-- Logs begin at Fri 2022-05-13 12:29:39 UTC, end at Fri 2022-05-13 13:43:07 UTC. -- May 13 13:02:51.479366 dv-cinder-841591-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 13 13:02:51.488589 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 13 13:02:51 2022] *** May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: nodename: dv-cinder-841591-2-iscsi May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: machine: x86_64 May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: clock source: unix May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: pcre jit disabled May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: detected number of CPU cores: 4 May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: current working directory: / May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: detected binary path: /usr/bin/uwsgi-core May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: your processes number limit is 47680 May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: your memory page size is 4096 bytes May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: detected max file descriptor number: 2048 May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: lock engine: pthread robust mutexes May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: thunder lock: enabled May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 13 13:02:51.490972 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 13 13:02:51.536199 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: Python main interpreter initialized at 0x55dab6a479c0 May 13 13:02:51.536199 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: python threads support enabled May 13 13:02:51.536199 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: your server socket listen backlog is limited to 100 connections May 13 13:02:51.536199 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: your mercy for graceful operations on workers is 90 seconds May 13 13:02:51.536574 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: mapped 403077 bytes (393 KB) for 2 cores May 13 13:02:51.536651 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: *** Operational MODE: preforking *** May 13 13:02:51.536892 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: *** uWSGI is running in multiple interpreter mode *** May 13 13:02:51.536892 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: spawned uWSGI master process (pid: 80262) May 13 13:02:51.537092 dv-cinder-841591-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 13 13:02:51.539415 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: spawned uWSGI worker 1 (pid: 80277, cores: 1) May 13 13:02:51.539415 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: spawned uWSGI worker 2 (pid: 80278, cores: 1) May 13 13:02:51.539415 dv-cinder-841591-2-iscsi devstack@keystone.service[80262]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 13 13:02:57.317393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 13:02:57.317393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: from cryptography.utils import int_from_bytes May 13 13:02:57.318179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 13:02:57.318179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: from cryptography.utils import int_from_bytes May 13 13:02:57.318291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 13:02:57.318291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: from cryptography.utils import int_from_bytes May 13 13:02:57.318291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 13 13:02:57.318291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: from cryptography.utils import int_from_bytes May 13 13:02:57.366130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 13 13:02:57.366310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 13 13:02:57.366394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 13 13:02:57.366482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] command line args: [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 13 13:02:57.366565 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 13 13:02:57.366686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] command line args: [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 13 13:02:57.366763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 13 13:02:57.366910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ================================================================================ {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 13 13:02:57.366986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 13 13:02:57.367121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ================================================================================ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 13 13:02:57.367206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] admin_token = **** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.367378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] admin_token = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.367456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] config_dir = [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.367560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.367644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_dir = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.367727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] config_source = [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.367804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.367902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] debug = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.367991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] config_source = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] debug = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368509 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.368975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] list_limit = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] log_config_append = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_config_append = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.369987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] log_dir = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_dir = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] log_file = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370304 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.370926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371799 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.371956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.372072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.372189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.372303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.372422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.372536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.372654 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.372770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.372900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.373017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.373143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.373261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.373415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.373539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.373919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] publish_errors = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] publish_errors = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.374931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375166 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] standard_threads = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] standard_threads = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375873 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.375960 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] use_journal = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] use_json = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_journal = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] use_stderr = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_json = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] use_syslog = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_stderr = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] use_syslog = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.376906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.377062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 13 13:02:57.377148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.377308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.377385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.377543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.377619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.377800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.377895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.378055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.378134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.378291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.378375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.378530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.378615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.378763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.378839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.379964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.380983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.381141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.381223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.381393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.381482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.381633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.381716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.381894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.381986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.382139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.382224 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.382391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.382468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.382639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.382716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.382888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.382968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.383119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.383196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.383347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.383424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.383575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.383653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.383803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.383879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.384968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.385072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.385215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.385318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.385451 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.385553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.385686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.385887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.385976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386853 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.386935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.387079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.387162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.387307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.387384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.387533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.388104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.388286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.388371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.388514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.388593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.388751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.388835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.388979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.389066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.389207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.389283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.389436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.389514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.389668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.389753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.389943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.390929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.391075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.391161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.391303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.391389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.391530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.391650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.391767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.391854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.392994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.393151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.393238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.393384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.393463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.393617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.393700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.393905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.394000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.394145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.394266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.394499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.394578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.394740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.394850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.394976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.395056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.395212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.395296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.395451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.395541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.395687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.395774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.395925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396403 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396873 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.396950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.397107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.397193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.397339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.397416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.397574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.397658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.397843 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.397925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.398086 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.398175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.398324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.398403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.398560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.398645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.398796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.398874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.399956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.400952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.401108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.401191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.401346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.401433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.401581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.401660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.401859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.401943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.402100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.402180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.402331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] auth.external = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.402409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.402560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.402639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.external = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.402790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.402869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.403075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.403164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.403363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.403976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] auth.password = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.404111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.password = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.404212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] auth.token = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.404347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] auth.token = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.404453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.404588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.404731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.404809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.404922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.405058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.405139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.405292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.405371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.405540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.405620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.405832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.405919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.406043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.406123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] credential.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.406281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.406360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.406529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.406612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.406779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.406859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407798 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.407983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.408064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.408217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.408295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.408467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.408547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.408717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.408803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.408981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.409068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.409234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.409312 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.409472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.409553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.409727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.409826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.409987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] federation.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.410066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.410225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.410306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.410461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.410539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.410704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.410783 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.410935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411245 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.411964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.412121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.412208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.412356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.412435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.412602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.412682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.412835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.412922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.413073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.413151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.413316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.413395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.413552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.413640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.413815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.413898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.414996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.415082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.415230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.415310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.415460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.415540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.415690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.415776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.415922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416736 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.416985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.417144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.417231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.417391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.417477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.417629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.417718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.417878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.417973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.418114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.418193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.418342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.418422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.418573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.418663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.418809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.418889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.419070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.419149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.419317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.419405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.419546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.419632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.419777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.419856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.420958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.421988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.422133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.422212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.422365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.422444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.422600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.422680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.422834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.422920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.423078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.423157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.423307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.423388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.423535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.423616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.423764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.423842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424786 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.424938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425723 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.425988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.426155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.426232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.426408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.426492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.426625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.426703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.426852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.426931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427618 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.427992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.428070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.428241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.428319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.428483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.428562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.428728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.428807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.428970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.429048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.429195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.429272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.429427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.429505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.429661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.429739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.429919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.430937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.431096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.431175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.431330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.431409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.431563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.431640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.431799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.431876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.432032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.432111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.432264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.432597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.432752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.432829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.432989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.433070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.433237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.433317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.433455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.433535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.433683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.433769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.433939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] resource.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.434020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.434179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.434258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.434422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.434502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.435171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.435258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.435408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.435486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.435652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.435732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.435885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.435965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.436124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.436363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.436442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.436597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.436675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.436828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] role.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.436905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.437060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.437143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] role.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.437306 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.437411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.437533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.437621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.437793 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.437884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.438050 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.438128 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.438297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.438387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.438541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.438619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.438785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.438863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.439977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.440055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.440219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.440297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.440462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.440548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.440706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.440859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.441097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.441174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.441333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.441418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.441580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.441657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.441845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.441930 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.442095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.442185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.442327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.442404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.442567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.442645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.442816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.442895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.443971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.444048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.444223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.444303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.444464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.444542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.444698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.444777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.444931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] token.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.445960 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.446114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.446203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.446353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.446432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.446604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.446681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.446838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.446917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.447080 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.447169 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.447307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.447384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.447546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.447623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.447779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.447858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448245 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.448947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.449025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.449182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.449261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.449427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.449505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.449721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.449819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.connection = **** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.449991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.450083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.450292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.450394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.450536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.450615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.450774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.450851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.451944 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.453111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.453197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.453350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.453427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.453575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.453653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.453832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.453929 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.454069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.454149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.454221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80278) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 13 13:02:57.454292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 13 13:02:57.454457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 13 13:02:57.485749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.486005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.486097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.486226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.486334 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.486464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.494920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.495161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.523204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.523435 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.523605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.523837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.576425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.577646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.627419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.627646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.628177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.628414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.693122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.693247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.693335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.693421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.693562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.693647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.693805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.693903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.694029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.694120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.694245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.694327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.717629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.717840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.717936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.718940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.719063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.719148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.730173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.730339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.744793 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.744916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.745918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.746014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.746130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80278) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.746212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80277) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 13 13:02:57.764202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.764411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 13:02:57.764511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.764625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 13:02:57.764746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.764947 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 13:02:57.765029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.765144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 13 13:02:57.765235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.765411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.765495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.765667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.765748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.765950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.766035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.766201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.776843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.777046 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.777131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.777319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.777403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.777580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.777662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.777879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.785705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.785927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.789018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.789182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.789297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.789455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.789579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.789732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.789995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.790142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.790280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.790422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.790543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.790682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.790815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.790950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.791090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.791220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.791362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.791485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.791637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.791753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.791905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.792015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.819009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.819228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.820599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.820872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.820955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.821150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.826680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.826910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.827002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.827187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.827271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.827528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.834424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.834701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.834874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.835154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.839277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.839558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.839771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.839858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.840058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.840141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.840339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.840422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.840607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.840690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.840924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.841008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.841189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.841282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.841455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.841722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.860322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.860596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.860759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.861036 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.865117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.865524 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.868746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.869038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.869133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.869413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.874084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.874419 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.877645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.877975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.881389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.881663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.881747 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.881974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.882069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.882258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.882350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.882527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.882618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.882799 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.882883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.883076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.902157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.902526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.902646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.902791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.903020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.903104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.903848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.904116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.911425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.911963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.913136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.913409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.913683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.913981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.916444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.916717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.916994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.917088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.917274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.917358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.917548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.917632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.917854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.918137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.931231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.931479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.931604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.931746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.931871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.932024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.932109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.932548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.932659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.932843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.932926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.933122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.933206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.933394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.951865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.951967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.955139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.955226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.958436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.958541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.960039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.960124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.961645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.961728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.961942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.962040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.962233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.962316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.969626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.969748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.971236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.971367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.974449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.974606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.974737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.974897 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.975014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.975183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.975293 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.975459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.984958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.985187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.985272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.985476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.985560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.985758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.993532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80278) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.993688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80277) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 13 13:02:57.993822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.993992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.994143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.994273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.994402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.994536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.994669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.994814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.994955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.995088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.995222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.995355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.995441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.995589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.995724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.995854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.996000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.996132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.996283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.996413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.996542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.996672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.996800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.996958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.997658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80278) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:57.997814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80277) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 13 13:02:58.035681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55dab6a479c0 pid: 80278 (default app) May 13 13:02:58.036045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55dab6a479c0 pid: 80277 (default app) May 13 13:02:58.039008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2e16f-2c4b-418e-9e65-ff7d8f94c091 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:02:58.039308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2e16f-2c4b-418e-9e65-ff7d8f94c091 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:02:58.039592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2e16f-2c4b-418e-9e65-ff7d8f94c091 None None] PATH_INFO: `/v3` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:02:58.043702 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Fri May 13 13:02:58 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:07.049450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bda6a-6c14-45be-80da-c741ef9eaa8a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:07.050046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bda6a-6c14-45be-80da-c741ef9eaa8a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:07.050046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bda6a-6c14-45be-80da-c741ef9eaa8a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:07.054368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:07.060442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:07.060915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:07.061101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:07.092255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG oslo_db.sqlalchemy.engines [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80278) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 13 13:03:07.280217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG passlib.handlers.bcrypt [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80278) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 13 13:03:07.280640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG passlib.handlers.bcrypt [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80278) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 13 13:03:07.338102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:07.339636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:07.458438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 13 13:03:07.649093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bda6a-6c14-45be-80da-c741ef9eaa8a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:07.683088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG oslo_db.sqlalchemy.engines [None req-179bda6a-6c14-45be-80da-c741ef9eaa8a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80277) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 13 13:03:07.807769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bda6a-6c14-45be-80da-c741ef9eaa8a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:07.812048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa646f-ea68-4fc9-a6b9-1a3cc2c72a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:07.816494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa646f-ea68-4fc9-a6b9-1a3cc2c72a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:07.816494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa646f-ea68-4fc9-a6b9-1a3cc2c72a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:07.816494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa646f-ea68-4fc9-a6b9-1a3cc2c72a90 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:07.821114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa646f-ea68-4fc9-a6b9-1a3cc2c72a90 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:07.872891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1bfa646f-ea68-4fc9-a6b9-1a3cc2c72a90 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 13 13:03:07.873228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa646f-ea68-4fc9-a6b9-1a3cc2c72a90 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:07.882467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Fri May 13 13:03:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:03:07.888837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:07.902924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc2296-7c57-4288-858a-eaea0cfa4996 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:07.907875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c398d2-f34e-4e54-b503-3902899b2566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:07.908989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c398d2-f34e-4e54-b503-3902899b2566 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:07.909280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c398d2-f34e-4e54-b503-3902899b2566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:07.909559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c398d2-f34e-4e54-b503-3902899b2566 admin admin] PATH_INFO: `/v3/projects/989de7565a074e299634e689ed15d9f9` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:07.911152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c398d2-f34e-4e54-b503-3902899b2566 admin admin] RBAC: Authorizing `identity:get_project(project_id=989de7565a074e299634e689ed15d9f9)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:07.961723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c2c398d2-f34e-4e54-b503-3902899b2566 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 13 13:03:07.962093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c398d2-f34e-4e54-b503-3902899b2566 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:07.966598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Fri May 13 13:03:07 2022] GET /identity/v3/projects/989de7565a074e299634e689ed15d9f9 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:10.128835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be4e0b-4902-4dcb-89b3-690a15b2b61c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:10.129376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be4e0b-4902-4dcb-89b3-690a15b2b61c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:10.129376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be4e0b-4902-4dcb-89b3-690a15b2b61c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:10.133455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:10.139353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c237ab-8ef7-4f28-ba6c-021db9f90b64 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:10.139780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c237ab-8ef7-4f28-ba6c-021db9f90b64 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:10.139920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c237ab-8ef7-4f28-ba6c-021db9f90b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:10.159838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f8c237ab-8ef7-4f28-ba6c-021db9f90b64 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:10.161348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f8c237ab-8ef7-4f28-ba6c-021db9f90b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:10.169135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 13 13:03:10.191232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be4e0b-4902-4dcb-89b3-690a15b2b61c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:10.203599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be4e0b-4902-4dcb-89b3-690a15b2b61c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:10.207843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732517a-15e5-4472-bdd8-9eafb63cce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:10.208467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732517a-15e5-4472-bdd8-9eafb63cce7d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:10.208755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732517a-15e5-4472-bdd8-9eafb63cce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:10.209038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732517a-15e5-4472-bdd8-9eafb63cce7d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:10.216618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732517a-15e5-4472-bdd8-9eafb63cce7d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:10.218444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732517a-15e5-4472-bdd8-9eafb63cce7d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:10.227118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Fri May 13 13:03:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:03:10.232452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c237ab-8ef7-4f28-ba6c-021db9f90b64 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:10.245425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c237ab-8ef7-4f28-ba6c-021db9f90b64 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:10.252701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf23e4-296f-422a-aa79-451138e60512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:10.254344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf23e4-296f-422a-aa79-451138e60512 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:10.254825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf23e4-296f-422a-aa79-451138e60512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:10.255296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf23e4-296f-422a-aa79-451138e60512 admin admin] PATH_INFO: `/v3/users/df2ff43339cf407eb4cebe6484112e01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:10.256734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf23e4-296f-422a-aa79-451138e60512 admin admin] RBAC: Authorizing `identity:get_user(user_id=df2ff43339cf407eb4cebe6484112e01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:10.258641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf23e4-296f-422a-aa79-451138e60512 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:10.260301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Fri May 13 13:03:10 2022] GET /identity/v3/users/df2ff43339cf407eb4cebe6484112e01 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:12.504195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1631a4-0dbc-4b66-81f5-ae3d9591c143 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:12.504688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1631a4-0dbc-4b66-81f5-ae3d9591c143 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:12.504782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1631a4-0dbc-4b66-81f5-ae3d9591c143 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:12.506117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:12.511318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4fa6a-6a29-4fbc-b3b0-1d567c165e5e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:12.511721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4fa6a-6a29-4fbc-b3b0-1d567c165e5e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:12.511965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4fa6a-6a29-4fbc-b3b0-1d567c165e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:12.552227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26fed-9ded-47b7-8b86-8f3bd71af788 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:12.552518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26fed-9ded-47b7-8b86-8f3bd71af788 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:12.552804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26fed-9ded-47b7-8b86-8f3bd71af788 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:12.554214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:12.560411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:12.560706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:12.560987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:12.603168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG passlib.handlers.bcrypt [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80277) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 13 13:03:12.603606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG passlib.handlers.bcrypt [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80277) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 13 13:03:12.636639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-79e4fa6a-6a29-4fbc-b3b0-1d567c165e5e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:12.637920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-79e4fa6a-6a29-4fbc-b3b0-1d567c165e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:12.650200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:12.651775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:12.715047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:12.738382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4fa6a-6a29-4fbc-b3b0-1d567c165e5e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:12.751966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:12.773470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:12.833427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4fa6a-6a29-4fbc-b3b0-1d567c165e5e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:12.865689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ab686-6722-40ed-908d-f810e50481e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:12.866534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab686-6722-40ed-908d-f810e50481e3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:12.866831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab686-6722-40ed-908d-f810e50481e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:12.867109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab686-6722-40ed-908d-f810e50481e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:12.869473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6891073-7024-4800-8d63-8535d4bb0147 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:12.877873 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ab686-6722-40ed-908d-f810e50481e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-147ab686-6722-40ed-908d-f810e50481e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:12.889524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:12.890480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:12.891409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:12.896411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ab686-6722-40ed-908d-f810e50481e3 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:12.903627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510722b2-832b-45d4-a5e0-51d36ef74811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:12.904533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510722b2-832b-45d4-a5e0-51d36ef74811 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:12.904819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510722b2-832b-45d4-a5e0-51d36ef74811 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:12.905096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510722b2-832b-45d4-a5e0-51d36ef74811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:12.976551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510722b2-832b-45d4-a5e0-51d36ef74811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:12.978104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510722b2-832b-45d4-a5e0-51d36ef74811 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:13.000564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ab686-6722-40ed-908d-f810e50481e3 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:13.032668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:13.033332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:13.033620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:13.033938 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:13.034685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:13.034992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:13.036370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:13.047573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:13.052963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:13.069418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:03:13.075332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510722b2-832b-45d4-a5e0-51d36ef74811 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:13.146249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866028a8-1b3b-4a99-a6b9-97d860d3cbb7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:13.168412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510722b2-832b-45d4-a5e0-51d36ef74811 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:13.179868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c5978-39a3-47fc-aca6-7f4948078e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:13.180737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c5978-39a3-47fc-aca6-7f4948078e74 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:13.181024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c5978-39a3-47fc-aca6-7f4948078e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:13.181297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c5978-39a3-47fc-aca6-7f4948078e74 None admin] PATH_INFO: `/v3/users/df2ff43339cf407eb4cebe6484112e01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:13.182864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c5978-39a3-47fc-aca6-7f4948078e74 None admin] RBAC: Authorizing `identity:get_user(user_id=df2ff43339cf407eb4cebe6484112e01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:13.184348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c5978-39a3-47fc-aca6-7f4948078e74 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:13.185977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:13 2022] GET /identity/v3/users/df2ff43339cf407eb4cebe6484112e01 => generated 247 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:13.191354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c5978-39a3-47fc-aca6-7f4948078e74 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:13.200860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:13.201659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:13.201985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:13.202267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:13.289336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c5978-39a3-47fc-aca6-7f4948078e74 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:13.323564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b115c-3ba4-44d6-a022-bbf3f619a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:13.324362 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b115c-3ba4-44d6-a022-bbf3f619a72c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:13.324667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b115c-3ba4-44d6-a022-bbf3f619a72c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:13.324953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b115c-3ba4-44d6-a022-bbf3f619a72c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:13.326288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b115c-3ba4-44d6-a022-bbf3f619a72c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:13.327651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:13.327772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b115c-3ba4-44d6-a022-bbf3f619a72c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:13.329106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:13.329379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:13.335117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b115c-3ba4-44d6-a022-bbf3f619a72c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:03:13.350238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:13.351098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:13.351910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:03:13.351910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:03:13.351910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:03:13.351910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:13.351910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:13.357121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:13.427864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b115c-3ba4-44d6-a022-bbf3f619a72c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:13.451371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d90077-8e97-4443-a276-844d4d7b60d4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:13.459260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d835155-e129-4396-93e2-1ab41a8c58bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:13.459909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d835155-e129-4396-93e2-1ab41a8c58bd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:13.460206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d835155-e129-4396-93e2-1ab41a8c58bd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:13.460481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d835155-e129-4396-93e2-1ab41a8c58bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:13.461307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d835155-e129-4396-93e2-1ab41a8c58bd None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.domain.id=default, user.id=df2ff43339cf407eb4cebe6484112e01) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:13.461622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d835155-e129-4396-93e2-1ab41a8c58bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:13.463102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d835155-e129-4396-93e2-1ab41a8c58bd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:13.470089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Fri May 13 13:03:13 2022] GET /identity/v3/role_assignments?user.id=df2ff43339cf407eb4cebe6484112e01&scope.domain.id=default&role.id=663341777b0b44128ed78fe8d40f6462 => generated 233 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:13.485230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a129eec-e0c4-43ba-ad57-b9b0a4ccebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:13.486070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a129eec-e0c4-43ba-ad57-b9b0a4ccebf8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:13.486357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a129eec-e0c4-43ba-ad57-b9b0a4ccebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:13.486636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a129eec-e0c4-43ba-ad57-b9b0a4ccebf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:13.487387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a129eec-e0c4-43ba-ad57-b9b0a4ccebf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:13.487696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a129eec-e0c4-43ba-ad57-b9b0a4ccebf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:13.489152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a129eec-e0c4-43ba-ad57-b9b0a4ccebf8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:13.508813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:16.581828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb19f19-168f-4550-86ef-3c6871fb4279 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:16.581828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb19f19-168f-4550-86ef-3c6871fb4279 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:16.581828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb19f19-168f-4550-86ef-3c6871fb4279 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:16.583605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:16.591530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbfd60-21e8-40cc-86ec-b93bcffb935e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:16.592834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbfd60-21e8-40cc-86ec-b93bcffb935e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:16.593603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbfd60-21e8-40cc-86ec-b93bcffb935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:16.622143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-43fbfd60-21e8-40cc-86ec-b93bcffb935e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:16.624494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-43fbfd60-21e8-40cc-86ec-b93bcffb935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:16.727030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780ebb6-9557-4599-8797-9a4a6d91ab0f None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:16.727911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780ebb6-9557-4599-8797-9a4a6d91ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:16.728640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780ebb6-9557-4599-8797-9a4a6d91ab0f None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:16.739965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 13/23] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:16 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:16.745749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f63c3-afb4-48e0-b10c-a7fd46cebcc5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:16.746581 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f63c3-afb4-48e0-b10c-a7fd46cebcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:16.747329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f63c3-afb4-48e0-b10c-a7fd46cebcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:16.749055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 11/24] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:16.791133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbfd60-21e8-40cc-86ec-b93bcffb935e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:16.798271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-188f63c3-afb4-48e0-b10c-a7fd46cebcc5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:16.798271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-188f63c3-afb4-48e0-b10c-a7fd46cebcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:16.977618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbfd60-21e8-40cc-86ec-b93bcffb935e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:17.016545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 14/25] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:17.018035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041143-c2b0-460e-aa93-329c9211e23b None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.018379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041143-c2b0-460e-aa93-329c9211e23b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.018668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041143-c2b0-460e-aa93-329c9211e23b None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.019940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 15/26] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:17.025455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9683b-c7b1-497a-8f9e-cfb65f980c91 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.025742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9683b-c7b1-497a-8f9e-cfb65f980c91 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.026223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9683b-c7b1-497a-8f9e-cfb65f980c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.043775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:17.046693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.047482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.048286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.067636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:17.070711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:17.070864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-88f9683b-c7b1-497a-8f9e-cfb65f980c91 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:17.072022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-88f9683b-c7b1-497a-8f9e-cfb65f980c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:03:17.108441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:17.110050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:17.111657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:03:17.111657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:03:17.111657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:03:17.111657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:17.116186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 12/27] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:17.122354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:17.180772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 16/28] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:17.181615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9683b-c7b1-497a-8f9e-cfb65f980c91 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:17.281039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9683b-c7b1-497a-8f9e-cfb65f980c91 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:17.306437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94f916-4da2-45f3-b201-b142a74b21e0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:17.318527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6699e0-3008-40ba-abde-58f8018b4518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:17.319527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6699e0-3008-40ba-abde-58f8018b4518 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.319959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6699e0-3008-40ba-abde-58f8018b4518 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.320381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6699e0-3008-40ba-abde-58f8018b4518 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.321256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6699e0-3008-40ba-abde-58f8018b4518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:17.321706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6699e0-3008-40ba-abde-58f8018b4518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:17.323279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6699e0-3008-40ba-abde-58f8018b4518 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:17.356418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 17/29] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:17.358478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7878a-186e-4d38-8c22-864464b26505 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.358995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7878a-186e-4d38-8c22-864464b26505 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.359464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7878a-186e-4d38-8c22-864464b26505 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.361457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 18/30] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:17.367230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7878a-186e-4d38-8c22-864464b26505 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:17.371706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd732dc-81d8-4268-bb72-94dc1da51edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:17.373181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd732dc-81d8-4268-bb72-94dc1da51edb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.373892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd732dc-81d8-4268-bb72-94dc1da51edb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.374471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd732dc-81d8-4268-bb72-94dc1da51edb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.396780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd732dc-81d8-4268-bb72-94dc1da51edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8fd732dc-81d8-4268-bb72-94dc1da51edb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:17.415381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:17.416239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:17.416981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:17.422351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 13/31] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:17.422351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd732dc-81d8-4268-bb72-94dc1da51edb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:17.488203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7878a-186e-4d38-8c22-864464b26505 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:17.536641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472d5c1-7b7b-41e3-a5ec-2e9ef8f72b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:17.537720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d5c1-7b7b-41e3-a5ec-2e9ef8f72b00 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.538270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d5c1-7b7b-41e3-a5ec-2e9ef8f72b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.538787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d5c1-7b7b-41e3-a5ec-2e9ef8f72b00 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.540232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472d5c1-7b7b-41e3-a5ec-2e9ef8f72b00 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:17.542923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472d5c1-7b7b-41e3-a5ec-2e9ef8f72b00 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:17.580814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd732dc-81d8-4268-bb72-94dc1da51edb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:17.621623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3137550-6c23-4a60-8e03-b2b9e49e71c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:17.622309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137550-6c23-4a60-8e03-b2b9e49e71c7 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.622604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137550-6c23-4a60-8e03-b2b9e49e71c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.622885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137550-6c23-4a60-8e03-b2b9e49e71c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.623844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3137550-6c23-4a60-8e03-b2b9e49e71c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:17.625330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3137550-6c23-4a60-8e03-b2b9e49e71c7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:17.765055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 19/32] 10.222.0.25 () {60 vars in 1131 bytes} [Fri May 13 13:03:17 2022] POST /identity/v3/roles => generated 219 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:17.767270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319fee-fbd0-4b50-beb8-b986ac64feb1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.767769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319fee-fbd0-4b50-beb8-b986ac64feb1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:17.768274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319fee-fbd0-4b50-beb8-b986ac64feb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:17.787968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-31319fee-fbd0-4b50-beb8-b986ac64feb1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:17.789972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-31319fee-fbd0-4b50-beb8-b986ac64feb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:17.829831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 14/33] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:03:17 2022] POST /identity/v3/projects => generated 301 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:17.829831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3137550-6c23-4a60-8e03-b2b9e49e71c7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:17.891758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 20/34] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:17.938173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31319fee-fbd0-4b50-beb8-b986ac64feb1 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:17.949858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3137550-6c23-4a60-8e03-b2b9e49e71c7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:17.998546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:17.999500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:17.999972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.000403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.001333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:18.001824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:18.003541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:18.024095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 15/35] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:18.029838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:18.160652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31319fee-fbd0-4b50-beb8-b986ac64feb1 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:18.195258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3fe8-1530-4091-9237-ef89eb33204d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:18.211198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84cc9c-b1fc-4f87-ba7a-6c30d1caf2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:18.212265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84cc9c-b1fc-4f87-ba7a-6c30d1caf2c3 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.213880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84cc9c-b1fc-4f87-ba7a-6c30d1caf2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.214363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84cc9c-b1fc-4f87-ba7a-6c30d1caf2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.215904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84cc9c-b1fc-4f87-ba7a-6c30d1caf2c3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:18.217610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84cc9c-b1fc-4f87-ba7a-6c30d1caf2c3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:18.239095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b0397-a9e1-4b70-8fa3-30650f562fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:18.240236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b0397-a9e1-4b70-8fa3-30650f562fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.240523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b0397-a9e1-4b70-8fa3-30650f562fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.240805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b0397-a9e1-4b70-8fa3-30650f562fc6 None admin] PATH_INFO: `/v3/users/df2ff43339cf407eb4cebe6484112e01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.242386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b0397-a9e1-4b70-8fa3-30650f562fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=df2ff43339cf407eb4cebe6484112e01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:18.243838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b0397-a9e1-4b70-8fa3-30650f562fc6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:18.245622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 16/36] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:18 2022] GET /identity/v3/users/df2ff43339cf407eb4cebe6484112e01 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:18.246896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d4aa3-7383-47ca-a581-cd562fb9b4d2 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.247088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d4aa3-7383-47ca-a581-cd562fb9b4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.247370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d4aa3-7383-47ca-a581-cd562fb9b4d2 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.248576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 17/37] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:18.253337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d4aa3-7383-47ca-a581-cd562fb9b4d2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:18.305357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 21/38] 10.222.0.25 () {60 vars in 1131 bytes} [Fri May 13 13:03:17 2022] POST /identity/v3/roles => generated 225 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:03:18.307038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b8c4-82bb-410e-a8ae-c6bfe39c8495 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.307795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b8c4-82bb-410e-a8ae-c6bfe39c8495 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.308297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b8c4-82bb-410e-a8ae-c6bfe39c8495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.327703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2e33b8c4-82bb-410e-a8ae-c6bfe39c8495 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:18.329391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2e33b8c4-82bb-410e-a8ae-c6bfe39c8495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:18.388845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d4aa3-7383-47ca-a581-cd562fb9b4d2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:18.416450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 22/39] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:18.417777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae276-1606-4c57-9b18-689009c00753 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.418118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae276-1606-4c57-9b18-689009c00753 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.418435 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ae276-1606-4c57-9b18-689009c00753 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.419912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 23/40] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:18.425524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b47876-a707-4c96-bf5e-f523c9e1fd7b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.425848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b47876-a707-4c96-bf5e-f523c9e1fd7b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.426140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b47876-a707-4c96-bf5e-f523c9e1fd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.432117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faffe8-3a39-406b-916d-ba0f7ba67ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:18.433337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faffe8-3a39-406b-916d-ba0f7ba67ffe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.433699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faffe8-3a39-406b-916d-ba0f7ba67ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.434619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faffe8-3a39-406b-916d-ba0f7ba67ffe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.436084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faffe8-3a39-406b-916d-ba0f7ba67ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:18.437637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faffe8-3a39-406b-916d-ba0f7ba67ffe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:18.439546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 18/41] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:18.445453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-86b47876-a707-4c96-bf5e-f523c9e1fd7b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:18.445914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faffe8-3a39-406b-916d-ba0f7ba67ffe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:18.447352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-86b47876-a707-4c96-bf5e-f523c9e1fd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:18.529221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 24/42] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:18.530164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b47876-a707-4c96-bf5e-f523c9e1fd7b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:18.564496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faffe8-3a39-406b-916d-ba0f7ba67ffe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:18.596629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872b1fa-cb7e-4cc4-81f0-445bf0849770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:18.597499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872b1fa-cb7e-4cc4-81f0-445bf0849770 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.597811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872b1fa-cb7e-4cc4-81f0-445bf0849770 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.598093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872b1fa-cb7e-4cc4-81f0-445bf0849770 None admin] PATH_INFO: `/v3/domains/default/users/df2ff43339cf407eb4cebe6484112e01/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.600093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872b1fa-cb7e-4cc4-81f0-445bf0849770 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=df2ff43339cf407eb4cebe6484112e01, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:18.601559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872b1fa-cb7e-4cc4-81f0-445bf0849770 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:18.632101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b47876-a707-4c96-bf5e-f523c9e1fd7b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:18.663486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef924c-7af3-40ac-91dd-3c06f4a0556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:18.664319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef924c-7af3-40ac-91dd-3c06f4a0556b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.664598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef924c-7af3-40ac-91dd-3c06f4a0556b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.664874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef924c-7af3-40ac-91dd-3c06f4a0556b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.666070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef924c-7af3-40ac-91dd-3c06f4a0556b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:18.667567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef924c-7af3-40ac-91dd-3c06f4a0556b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:18.669144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 25/43] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:18.669890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef924c-7af3-40ac-91dd-3c06f4a0556b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:18.696843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 19/44] 10.222.0.25 () {58 vars in 1361 bytes} [Fri May 13 13:03:18 2022] PUT /identity/v3/domains/default/users/df2ff43339cf407eb4cebe6484112e01/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:03:18.697714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872b1fa-cb7e-4cc4-81f0-445bf0849770 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:18.774984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef924c-7af3-40ac-91dd-3c06f4a0556b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:18.805907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872b1fa-cb7e-4cc4-81f0-445bf0849770 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:18.811522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292274c-cb38-4896-ba65-87df87cb3b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:18.812218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292274c-cb38-4896-ba65-87df87cb3b8d None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.812507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292274c-cb38-4896-ba65-87df87cb3b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.812788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292274c-cb38-4896-ba65-87df87cb3b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.813551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292274c-cb38-4896-ba65-87df87cb3b8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:18.815028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292274c-cb38-4896-ba65-87df87cb3b8d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:18.842451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3c029-3dcf-4251-9187-31161de9abc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:18.843127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3c029-3dcf-4251-9187-31161de9abc5 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:18.843413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3c029-3dcf-4251-9187-31161de9abc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:18.843690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3c029-3dcf-4251-9187-31161de9abc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:18.844520 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3c029-3dcf-4251-9187-31161de9abc5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:18.846039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3c029-3dcf-4251-9187-31161de9abc5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:18.875986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 26/45] 10.222.0.25 () {60 vars in 1131 bytes} [Fri May 13 13:03:18 2022] POST /identity/v3/roles => generated 223 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:18.928725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 20/46] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:03:18 2022] POST /identity/v3/projects => generated 312 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:20.956482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263cc29-f407-4495-a367-00a1d58e1d8c None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:20.957033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263cc29-f407-4495-a367-00a1d58e1d8c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:20.957033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263cc29-f407-4495-a367-00a1d58e1d8c None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:20.958405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 27/47] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:20.964470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dfa46-449b-469a-ad34-709df7ab4707 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:20.965026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dfa46-449b-469a-ad34-709df7ab4707 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:20.965623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dfa46-449b-469a-ad34-709df7ab4707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:20.985048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-196dfa46-449b-469a-ad34-709df7ab4707 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:20.986684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-196dfa46-449b-469a-ad34-709df7ab4707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:21.051326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 21/48] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:21.077763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263cc29-f407-4495-a367-00a1d58e1d8c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:21.156294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47c94a-4a2d-4f2a-b7b6-47aba91e5a54 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.156586 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47c94a-4a2d-4f2a-b7b6-47aba91e5a54 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.156872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47c94a-4a2d-4f2a-b7b6-47aba91e5a54 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.158285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 22/49] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:21.163139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9613bb5-5137-4f32-941c-cdfe3b73b8a9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.163432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9613bb5-5137-4f32-941c-cdfe3b73b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.163722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9613bb5-5137-4f32-941c-cdfe3b73b8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.174872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263cc29-f407-4495-a367-00a1d58e1d8c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:21.180295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e9613bb5-5137-4f32-941c-cdfe3b73b8a9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:21.181474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e9613bb5-5137-4f32-941c-cdfe3b73b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:21.207113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6a0a-a103-45f4-9517-f23250c3e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:21.207851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6a0a-a103-45f4-9517-f23250c3e8fd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.208145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6a0a-a103-45f4-9517-f23250c3e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.208428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba6a0a-a103-45f4-9517-f23250c3e8fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.218165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba6a0a-a103-45f4-9517-f23250c3e8fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-56ba6a0a-a103-45f4-9517-f23250c3e8fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:21.225767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:21.226698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:21.227606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 28/50] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:21.233060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6a0a-a103-45f4-9517-f23250c3e8fd None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:21.244809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 23/51] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:21.267973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9613bb5-5137-4f32-941c-cdfe3b73b8a9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:21.326353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba6a0a-a103-45f4-9517-f23250c3e8fd None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:21.358102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:21.358769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.359060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.359308 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9613bb5-5137-4f32-941c-cdfe3b73b8a9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:21.359404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.360081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:21.360394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:21.361797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:21.369595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 29/52] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:21.375523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:21.391711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a60837-1807-436c-b85a-8b203bae01c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:21.392507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a60837-1807-436c-b85a-8b203bae01c9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.392794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a60837-1807-436c-b85a-8b203bae01c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.393079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a60837-1807-436c-b85a-8b203bae01c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.394278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a60837-1807-436c-b85a-8b203bae01c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:21.395728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a60837-1807-436c-b85a-8b203bae01c9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:21.397328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 24/53] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:21.403134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a60837-1807-436c-b85a-8b203bae01c9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:21.468165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dda702-2caa-4b13-a219-443dccfec961 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:21.494928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a60837-1807-436c-b85a-8b203bae01c9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:21.500559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40651b3-5af3-46ff-acfb-96b69fa9a44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:21.501437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40651b3-5af3-46ff-acfb-96b69fa9a44d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.501726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40651b3-5af3-46ff-acfb-96b69fa9a44d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.502036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40651b3-5af3-46ff-acfb-96b69fa9a44d None admin] PATH_INFO: `/v3/users/df2ff43339cf407eb4cebe6484112e01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.503252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40651b3-5af3-46ff-acfb-96b69fa9a44d None admin] RBAC: Authorizing `identity:get_user(user_id=df2ff43339cf407eb4cebe6484112e01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:21.504700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40651b3-5af3-46ff-acfb-96b69fa9a44d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:21.506524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 30/54] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:21 2022] GET /identity/v3/users/df2ff43339cf407eb4cebe6484112e01 => generated 247 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:21.511717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40651b3-5af3-46ff-acfb-96b69fa9a44d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:21.528466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b0162-5985-47d1-b13f-6e32f0480970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:21.529130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0162-5985-47d1-b13f-6e32f0480970 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.529418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0162-5985-47d1-b13f-6e32f0480970 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.529705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0162-5985-47d1-b13f-6e32f0480970 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.530453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b0162-5985-47d1-b13f-6e32f0480970 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:21.531941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b0162-5985-47d1-b13f-6e32f0480970 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:21.604339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40651b3-5af3-46ff-acfb-96b69fa9a44d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:21.606674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 25/55] 10.222.0.25 () {60 vars in 1140 bytes} [Fri May 13 13:03:21 2022] POST /identity/v3/projects => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:03:21.639453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145de99-bab6-46cf-af4d-f0e46c951ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:21.640277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145de99-bab6-46cf-af4d-f0e46c951ddd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.640564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145de99-bab6-46cf-af4d-f0e46c951ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.640842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145de99-bab6-46cf-af4d-f0e46c951ddd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.642121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145de99-bab6-46cf-af4d-f0e46c951ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:21.643613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145de99-bab6-46cf-af4d-f0e46c951ddd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:21.645428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 31/56] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:21.651385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b0162-5985-47d1-b13f-6e32f0480970 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:21.754085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b0162-5985-47d1-b13f-6e32f0480970 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:21.787709 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626745cc-8fbd-42de-9604-29608207294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:21.788378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626745cc-8fbd-42de-9604-29608207294d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:21.788667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626745cc-8fbd-42de-9604-29608207294d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:21.790092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626745cc-8fbd-42de-9604-29608207294d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:21.790092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626745cc-8fbd-42de-9604-29608207294d None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.domain.id=default, user.id=df2ff43339cf407eb4cebe6484112e01) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:21.790092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626745cc-8fbd-42de-9604-29608207294d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:21.791569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626745cc-8fbd-42de-9604-29608207294d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:21.800300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 26/57] 10.222.0.25 () {56 vars in 1322 bytes} [Fri May 13 13:03:21 2022] GET /identity/v3/role_assignments?user.id=df2ff43339cf407eb4cebe6484112e01&scope.domain.id=default&role.id=663341777b0b44128ed78fe8d40f6462 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:23.837040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294fc3e-d3ad-4cbd-92d1-4f272f7911f2 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:23.837548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294fc3e-d3ad-4cbd-92d1-4f272f7911f2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:23.837548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294fc3e-d3ad-4cbd-92d1-4f272f7911f2 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:23.838947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 32/58] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:23.844627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46387188-7066-4696-8725-883bca6c94e0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:23.845034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46387188-7066-4696-8725-883bca6c94e0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:23.845185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46387188-7066-4696-8725-883bca6c94e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:23.862034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-46387188-7066-4696-8725-883bca6c94e0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:23.863208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-46387188-7066-4696-8725-883bca6c94e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:23.923992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 27/59] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:23.948517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294fc3e-d3ad-4cbd-92d1-4f272f7911f2 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:24.044010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294fc3e-d3ad-4cbd-92d1-4f272f7911f2 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:24.075304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bcf50-fb36-43e5-b7ad-5fea98cd5be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:24.076090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bcf50-fb36-43e5-b7ad-5fea98cd5be8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:24.076371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bcf50-fb36-43e5-b7ad-5fea98cd5be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:24.076641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bcf50-fb36-43e5-b7ad-5fea98cd5be8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:24.077815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bcf50-fb36-43e5-b7ad-5fea98cd5be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:24.079268 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bcf50-fb36-43e5-b7ad-5fea98cd5be8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:24.080854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 33/60] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:24.086242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46387188-7066-4696-8725-883bca6c94e0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:24.178403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46387188-7066-4696-8725-883bca6c94e0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:24.209803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80b12f-5437-438d-a090-7902f588409e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:24.210478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80b12f-5437-438d-a090-7902f588409e None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:24.210761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80b12f-5437-438d-a090-7902f588409e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:24.211035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80b12f-5437-438d-a090-7902f588409e None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:24.211739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80b12f-5437-438d-a090-7902f588409e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:24.213173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80b12f-5437-438d-a090-7902f588409e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:24.296533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 28/61] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:03:24 2022] POST /identity/v3/users => generated 275 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:03:27.736385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8cb54-135a-4555-980b-dfeb097916e0 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:27.737360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8cb54-135a-4555-980b-dfeb097916e0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:27.737914 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8cb54-135a-4555-980b-dfeb097916e0 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:27.739520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 34/62] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:27.744816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9939a1b-6f58-48f7-a9f1-833fb0dab0e6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:27.745693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9939a1b-6f58-48f7-a9f1-833fb0dab0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:27.746729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9939a1b-6f58-48f7-a9f1-833fb0dab0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:27.770771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c9939a1b-6f58-48f7-a9f1-833fb0dab0e6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:27.772293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c9939a1b-6f58-48f7-a9f1-833fb0dab0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:27.796201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657815-29b4-4e88-a2a6-4e6474c2a2b6 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:27.796864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657815-29b4-4e88-a2a6-4e6474c2a2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:27.797354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657815-29b4-4e88-a2a6-4e6474c2a2b6 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:27.799039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 35/63] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:27.804980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2754ea-3dbe-42d6-ae22-0f61ce2dc407 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:27.805617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2754ea-3dbe-42d6-ae22-0f61ce2dc407 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:27.806148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2754ea-3dbe-42d6-ae22-0f61ce2dc407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:27.838462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bd2754ea-3dbe-42d6-ae22-0f61ce2dc407 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:27.840879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bd2754ea-3dbe-42d6-ae22-0f61ce2dc407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:27.892839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 29/64] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:27.915822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9939a1b-6f58-48f7-a9f1-833fb0dab0e6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:27.998931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 36/65] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:28.000301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965fe7-cf23-4b77-adb5-486832a76afa None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.000603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965fe7-cf23-4b77-adb5-486832a76afa None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.000884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965fe7-cf23-4b77-adb5-486832a76afa None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.002418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 37/66] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:28.019270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228639-c456-430a-afef-a87cdf907c7b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.019562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228639-c456-430a-afef-a87cdf907c7b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.020018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228639-c456-430a-afef-a87cdf907c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.024459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9939a1b-6f58-48f7-a9f1-833fb0dab0e6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.040079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-18228639-c456-430a-afef-a87cdf907c7b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:28.041584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-18228639-c456-430a-afef-a87cdf907c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:28.067185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d3f55-bc3d-4b40-9468-b9056a8aafa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.067993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d3f55-bc3d-4b40-9468-b9056a8aafa4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.068280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d3f55-bc3d-4b40-9468-b9056a8aafa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.068568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d3f55-bc3d-4b40-9468-b9056a8aafa4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.079253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d3f55-bc3d-4b40-9468-b9056a8aafa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ad5d3f55-bc3d-4b40-9468-b9056a8aafa4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:28.086921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:28.087763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:28.088511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 30/67] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:28.089698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d3f55-bc3d-4b40-9468-b9056a8aafa4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.119436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 38/68] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:28.120967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e8ab3-015d-4c0c-bebd-b879d83ff072 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.121335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e8ab3-015d-4c0c-bebd-b879d83ff072 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.121620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e8ab3-015d-4c0c-bebd-b879d83ff072 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.127175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 39/69] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:28.127175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e8ab3-015d-4c0c-bebd-b879d83ff072 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.202862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d3f55-bc3d-4b40-9468-b9056a8aafa4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.234599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0e240-2734-4d80-bb0c-c42cad7c30db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.235405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e240-2734-4d80-bb0c-c42cad7c30db None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.235693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e240-2734-4d80-bb0c-c42cad7c30db None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.235972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e240-2734-4d80-bb0c-c42cad7c30db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.236515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e8ab3-015d-4c0c-bebd-b879d83ff072 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.246002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0e240-2734-4d80-bb0c-c42cad7c30db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-95e0e240-2734-4d80-bb0c-c42cad7c30db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:28.254230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:28.255069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:28.256229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 31/70] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:28.259076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e48c72-33ce-4344-81a5-6528f5eb6f50 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.259368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e48c72-33ce-4344-81a5-6528f5eb6f50 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.259649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e48c72-33ce-4344-81a5-6528f5eb6f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.274370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.275045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.275332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.275611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.276358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:28.276450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e0e48c72-33ce-4344-81a5-6528f5eb6f50 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:28.276665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:28.277677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e0e48c72-33ce-4344-81a5-6528f5eb6f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:28.278111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:28.291540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 40/71] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:28.291540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.345764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 32/72] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:28.346635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e48c72-33ce-4344-81a5-6528f5eb6f50 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.387030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4f03-a971-4bca-b598-3a50894bfb7f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.418858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584d6cc-0fee-4bcb-8b33-dc7b748fce2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.419658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d6cc-0fee-4bcb-8b33-dc7b748fce2e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.419948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d6cc-0fee-4bcb-8b33-dc7b748fce2e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.420231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d6cc-0fee-4bcb-8b33-dc7b748fce2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.429824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584d6cc-0fee-4bcb-8b33-dc7b748fce2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-8584d6cc-0fee-4bcb-8b33-dc7b748fce2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:28.437717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:28.438679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:28.439659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:28.440637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e48c72-33ce-4344-81a5-6528f5eb6f50 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.440727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 41/73] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:28.441719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584d6cc-0fee-4bcb-8b33-dc7b748fce2e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.470967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44311c56-ddbb-443f-be5e-7e05864df968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.471642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44311c56-ddbb-443f-be5e-7e05864df968 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.471932 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44311c56-ddbb-443f-be5e-7e05864df968 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.472214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44311c56-ddbb-443f-be5e-7e05864df968 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.472975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44311c56-ddbb-443f-be5e-7e05864df968 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:28.473286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44311c56-ddbb-443f-be5e-7e05864df968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:28.474717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44311c56-ddbb-443f-be5e-7e05864df968 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:28.482857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 33/74] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:28.484108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a9dc6-d399-4461-b6b3-ba873349f7c7 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.484402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a9dc6-d399-4461-b6b3-ba873349f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.484686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a9dc6-d399-4461-b6b3-ba873349f7c7 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.486115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 34/75] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:28.490576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a9dc6-d399-4461-b6b3-ba873349f7c7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.538179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584d6cc-0fee-4bcb-8b33-dc7b748fce2e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.569281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4a4f-3792-4eb4-9cd9-1704756d389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.570189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4a4f-3792-4eb4-9cd9-1704756d389b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.570480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4a4f-3792-4eb4-9cd9-1704756d389b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.570762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4a4f-3792-4eb4-9cd9-1704756d389b None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.583819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4a4f-3792-4eb4-9cd9-1704756d389b None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:28.585164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a9dc6-d399-4461-b6b3-ba873349f7c7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.585281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4a4f-3792-4eb4-9cd9-1704756d389b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:28.586961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 42/76] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:28.587764 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4a4f-3792-4eb4-9cd9-1704756d389b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.616935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e256e-4461-472c-9322-dcc299d4a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.617740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e256e-4461-472c-9322-dcc299d4a14e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.618055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e256e-4461-472c-9322-dcc299d4a14e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.618335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e256e-4461-472c-9322-dcc299d4a14e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.627951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e256e-4461-472c-9322-dcc299d4a14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-653e256e-4461-472c-9322-dcc299d4a14e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:28.636018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:28.637423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:28.638439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 35/77] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:28.639098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e256e-4461-472c-9322-dcc299d4a14e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.684787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4a4f-3792-4eb4-9cd9-1704756d389b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.716638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c5405-26d8-4818-b52b-68acf1cd357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.717300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5405-26d8-4818-b52b-68acf1cd357e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.717589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5405-26d8-4818-b52b-68acf1cd357e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.717900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c5405-26d8-4818-b52b-68acf1cd357e None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.718642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c5405-26d8-4818-b52b-68acf1cd357e None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:28.718947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c5405-26d8-4818-b52b-68acf1cd357e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:28.720315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c5405-26d8-4818-b52b-68acf1cd357e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:28.727911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 43/78] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:28.729130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84036eb8-7497-4ce9-9492-4a0aa5776027 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.729426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84036eb8-7497-4ce9-9492-4a0aa5776027 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.729706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84036eb8-7497-4ce9-9492-4a0aa5776027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.735351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e256e-4461-472c-9322-dcc299d4a14e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.744920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-84036eb8-7497-4ce9-9492-4a0aa5776027 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:28.746129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-84036eb8-7497-4ce9-9492-4a0aa5776027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:28.767782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1693e-0252-4bfb-a79e-ccdfa8aadc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.768659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1693e-0252-4bfb-a79e-ccdfa8aadc85 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.768944 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1693e-0252-4bfb-a79e-ccdfa8aadc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.769222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1693e-0252-4bfb-a79e-ccdfa8aadc85 None admin] PATH_INFO: `/v3/users/df2ff43339cf407eb4cebe6484112e01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.770453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1693e-0252-4bfb-a79e-ccdfa8aadc85 None admin] RBAC: Authorizing `identity:get_user(user_id=df2ff43339cf407eb4cebe6484112e01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:28.771902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1693e-0252-4bfb-a79e-ccdfa8aadc85 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:28.773522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 36/79] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/users/df2ff43339cf407eb4cebe6484112e01 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:28.774319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1693e-0252-4bfb-a79e-ccdfa8aadc85 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.813238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 44/80] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:28.814894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84036eb8-7497-4ce9-9492-4a0aa5776027 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.869275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1693e-0252-4bfb-a79e-ccdfa8aadc85 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.900629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124be0d-6857-424d-a1c5-8989af19900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.901278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124be0d-6857-424d-a1c5-8989af19900b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.901563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124be0d-6857-424d-a1c5-8989af19900b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.901879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124be0d-6857-424d-a1c5-8989af19900b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.906886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84036eb8-7497-4ce9-9492-4a0aa5776027 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:28.938221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:28.938878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:28.939161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:28.939433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:28.940170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:28.940475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:28.941888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:28.949354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 45/81] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:28.950625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:28.965513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124be0d-6857-424d-a1c5-8989af19900b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:28.967025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124be0d-6857-424d-a1c5-8989af19900b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.044446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2f84-a622-44ff-bb92-07dd72949c98 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.059702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 37/82] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:29.060402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124be0d-6857-424d-a1c5-8989af19900b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.079799 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f110ace-84ba-460e-a559-2afc32126d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.080602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f110ace-84ba-460e-a559-2afc32126d7b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.080884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f110ace-84ba-460e-a559-2afc32126d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.081152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f110ace-84ba-460e-a559-2afc32126d7b None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.082365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f110ace-84ba-460e-a559-2afc32126d7b None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.083808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f110ace-84ba-460e-a559-2afc32126d7b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.085412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 46/83] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:28 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:29.086185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f110ace-84ba-460e-a559-2afc32126d7b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.159444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124be0d-6857-424d-a1c5-8989af19900b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.183476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f110ace-84ba-460e-a559-2afc32126d7b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.192956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd1476-4a9b-48cd-a8c0-39b56b643111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.193614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1476-4a9b-48cd-a8c0-39b56b643111 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.193937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1476-4a9b-48cd-a8c0-39b56b643111 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.194227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1476-4a9b-48cd-a8c0-39b56b643111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.218594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7dd8f-00d6-4c73-8bd7-6adcb778b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.219402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7dd8f-00d6-4c73-8bd7-6adcb778b94e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.219688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7dd8f-00d6-4c73-8bd7-6adcb778b94e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.219962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7dd8f-00d6-4c73-8bd7-6adcb778b94e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.221108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7dd8f-00d6-4c73-8bd7-6adcb778b94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.222576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7dd8f-00d6-4c73-8bd7-6adcb778b94e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.224175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 47/84] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:29.224930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7dd8f-00d6-4c73-8bd7-6adcb778b94e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.259427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd1476-4a9b-48cd-a8c0-39b56b643111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.260864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd1476-4a9b-48cd-a8c0-39b56b643111 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.316400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7dd8f-00d6-4c73-8bd7-6adcb778b94e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.348001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e991854-0120-4091-94a0-6b674f13a715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.348803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e991854-0120-4091-94a0-6b674f13a715 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.349091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e991854-0120-4091-94a0-6b674f13a715 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.349372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e991854-0120-4091-94a0-6b674f13a715 None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.350598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e991854-0120-4091-94a0-6b674f13a715 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.352027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e991854-0120-4091-94a0-6b674f13a715 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.352923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 38/85] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:29.358306 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 48/86] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:29.358443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd1476-4a9b-48cd-a8c0-39b56b643111 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.358549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e991854-0120-4091-94a0-6b674f13a715 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.445835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e991854-0120-4091-94a0-6b674f13a715 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.447470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd1476-4a9b-48cd-a8c0-39b56b643111 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.478339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fa136-fa04-459e-8174-d95b811fbb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.479000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fa136-fa04-459e-8174-d95b811fbb19 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.479291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fa136-fa04-459e-8174-d95b811fbb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.479570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fa136-fa04-459e-8174-d95b811fbb19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.479749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1923d-5509-46b5-9f53-f0dae054662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.480642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1923d-5509-46b5-9f53-f0dae054662f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.480925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1923d-5509-46b5-9f53-f0dae054662f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.481201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1923d-5509-46b5-9f53-f0dae054662f None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.482400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1923d-5509-46b5-9f53-f0dae054662f None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.483847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1923d-5509-46b5-9f53-f0dae054662f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.485688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 39/87] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:29.486276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1923d-5509-46b5-9f53-f0dae054662f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.543510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fa136-fa04-459e-8174-d95b811fbb19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.544987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fa136-fa04-459e-8174-d95b811fbb19 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.577972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1923d-5509-46b5-9f53-f0dae054662f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.610034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672305c-9790-4fd4-9eea-32c71fedf87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.610639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672305c-9790-4fd4-9eea-32c71fedf87e None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.610925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672305c-9790-4fd4-9eea-32c71fedf87e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.611205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672305c-9790-4fd4-9eea-32c71fedf87e None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.611907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672305c-9790-4fd4-9eea-32c71fedf87e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.613340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672305c-9790-4fd4-9eea-32c71fedf87e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.636735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 49/88] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:29.637575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fa136-fa04-459e-8174-d95b811fbb19 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.702730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 40/89] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:03:29 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:29.703552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672305c-9790-4fd4-9eea-32c71fedf87e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.734112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fa136-fa04-459e-8174-d95b811fbb19 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.768910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a537ee-341d-467e-bba5-e4c5471ca0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.769722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a537ee-341d-467e-bba5-e4c5471ca0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.769924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a537ee-341d-467e-bba5-e4c5471ca0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.770173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a537ee-341d-467e-bba5-e4c5471ca0b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.810009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672305c-9790-4fd4-9eea-32c71fedf87e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.842740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a537ee-341d-467e-bba5-e4c5471ca0b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.844258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a537ee-341d-467e-bba5-e4c5471ca0b7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.846399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d7c53-8209-482d-b361-178a128191d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:29.847273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d7c53-8209-482d-b361-178a128191d3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:29.847558 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d7c53-8209-482d-b361-178a128191d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:29.847832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d7c53-8209-482d-b361-178a128191d3 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:29.849048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d7c53-8209-482d-b361-178a128191d3 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:29.850547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d7c53-8209-482d-b361-178a128191d3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:29.852321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 41/90] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:29.853117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d7c53-8209-482d-b361-178a128191d3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:29.966838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d7c53-8209-482d-b361-178a128191d3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:29.975127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 50/91] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:29.975782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a537ee-341d-467e-bba5-e4c5471ca0b7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:30.012670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:30.013655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:30.014141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:30.014599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:30.015620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] RBAC: Adding query filter params (role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=1eafe2a57502441faa06c7ab6de43b01, user.id=1ff1944c64d841c3a8596ed53714d48d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:30.016150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:30.017627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:30.023461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 42/92] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/role_assignments?user.id=1ff1944c64d841c3a8596ed53714d48d&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=3552540dc0054d2e87fad160a358c58b => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:30.025772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:30.087868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a537ee-341d-467e-bba5-e4c5471ca0b7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:30.125276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734b965-fc49-4a1f-9dc4-dcedc892ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:30.126572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734b965-fc49-4a1f-9dc4-dcedc892ef6f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:30.127073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734b965-fc49-4a1f-9dc4-dcedc892ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:30.127638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734b965-fc49-4a1f-9dc4-dcedc892ef6f None admin] PATH_INFO: `/v3/projects/e54f643f23454f1d954d6649c0103db4` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:30.129209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734b965-fc49-4a1f-9dc4-dcedc892ef6f None admin] RBAC: Authorizing `identity:get_project(project_id=e54f643f23454f1d954d6649c0103db4)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:30.131008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734b965-fc49-4a1f-9dc4-dcedc892ef6f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:30.133055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 51/93] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:29 2022] GET /identity/v3/projects/e54f643f23454f1d954d6649c0103db4 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:30.134096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734b965-fc49-4a1f-9dc4-dcedc892ef6f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:30.136103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5e932-d07d-4b75-a90e-d298171bbf14 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:30.183543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:30.184296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:30.184582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:30.184859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:30.185713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=1eafe2a57502441faa06c7ab6de43b01, user.id=df2ff43339cf407eb4cebe6484112e01) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:30.186055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:30.187511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:30.202342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 43/94] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:30 2022] GET /identity/v3/role_assignments?user.id=df2ff43339cf407eb4cebe6484112e01&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=663341777b0b44128ed78fe8d40f6462 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:30.203387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:30.249825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734b965-fc49-4a1f-9dc4-dcedc892ef6f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:30.311386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f418ae-a563-4cd1-8826-c6038d53e629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:30.311386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f418ae-a563-4cd1-8826-c6038d53e629 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:30.311386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f418ae-a563-4cd1-8826-c6038d53e629 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:30.311386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f418ae-a563-4cd1-8826-c6038d53e629 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:30.311814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f418ae-a563-4cd1-8826-c6038d53e629 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:30.311814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f418ae-a563-4cd1-8826-c6038d53e629 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:30.311941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 52/95] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:30 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:30.321849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f418ae-a563-4cd1-8826-c6038d53e629 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:30.560006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c38144-23a2-43b9-816d-0ed854dcd032 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:30.612717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f418ae-a563-4cd1-8826-c6038d53e629 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:30.641527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc226d12-4c98-45f2-a90f-2942c0f59fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:30.646398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc226d12-4c98-45f2-a90f-2942c0f59fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:30.646936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc226d12-4c98-45f2-a90f-2942c0f59fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:30.647231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc226d12-4c98-45f2-a90f-2942c0f59fd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:30.648334 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc226d12-4c98-45f2-a90f-2942c0f59fd9 None admin] RBAC: Adding query filter params (role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=e54f643f23454f1d954d6649c0103db4, user.id=1ff1944c64d841c3a8596ed53714d48d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:30.648899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc226d12-4c98-45f2-a90f-2942c0f59fd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:30.650638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc226d12-4c98-45f2-a90f-2942c0f59fd9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:30.656918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 44/96] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:30 2022] GET /identity/v3/role_assignments?user.id=1ff1944c64d841c3a8596ed53714d48d&scope.project.id=e54f643f23454f1d954d6649c0103db4&role.id=3552540dc0054d2e87fad160a358c58b => generated 259 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:30.671198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8bd52-b0b1-43d1-944e-4656f4ef3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:30.671895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bd52-b0b1-43d1-944e-4656f4ef3a1f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:30.672186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bd52-b0b1-43d1-944e-4656f4ef3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:30.672473 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bd52-b0b1-43d1-944e-4656f4ef3a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:30.673322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8bd52-b0b1-43d1-944e-4656f4ef3a1f None admin] RBAC: Adding query filter params (role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=1eafe2a57502441faa06c7ab6de43b01, user.id=1ff1944c64d841c3a8596ed53714d48d) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:30.673701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8bd52-b0b1-43d1-944e-4656f4ef3a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:30.675191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8bd52-b0b1-43d1-944e-4656f4ef3a1f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:30.686821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 53/97] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:30 2022] GET /identity/v3/role_assignments?user.id=1ff1944c64d841c3a8596ed53714d48d&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=9065c36fc305418baa5b977858b445dd => generated 259 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:32.938095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489e60-86ed-4bbb-8ec7-d3ce8888c657 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:32.939206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489e60-86ed-4bbb-8ec7-d3ce8888c657 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:32.939729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489e60-86ed-4bbb-8ec7-d3ce8888c657 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:32.941420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 45/98] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:32.947566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d07d8-8e16-4fdd-be13-e2491b9ba9e5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:32.948575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d07d8-8e16-4fdd-be13-e2491b9ba9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:32.949073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d07d8-8e16-4fdd-be13-e2491b9ba9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:32.967816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d95d07d8-8e16-4fdd-be13-e2491b9ba9e5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:32.969434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d95d07d8-8e16-4fdd-be13-e2491b9ba9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:33.056418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 54/99] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:33.089562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489e60-86ed-4bbb-8ec7-d3ce8888c657 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:33.216149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489e60-86ed-4bbb-8ec7-d3ce8888c657 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:33.257289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653901f-fd44-4063-bc7a-edc83b96efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:33.258130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653901f-fd44-4063-bc7a-edc83b96efc2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.258420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653901f-fd44-4063-bc7a-edc83b96efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.258703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653901f-fd44-4063-bc7a-edc83b96efc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.274790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653901f-fd44-4063-bc7a-edc83b96efc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a653901f-fd44-4063-bc7a-edc83b96efc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:33.283281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:33.284169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:33.284972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 46/100] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:33.291518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d07d8-8e16-4fdd-be13-e2491b9ba9e5 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:33.345909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dc964-8830-4b64-b0f4-72b942268c69 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.346243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dc964-8830-4b64-b0f4-72b942268c69 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.346494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dc964-8830-4b64-b0f4-72b942268c69 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.348143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 47/101] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:33.357161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9312b7f-5ea6-4715-a3fb-383ccf856c5b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.357459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9312b7f-5ea6-4715-a3fb-383ccf856c5b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.358124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9312b7f-5ea6-4715-a3fb-383ccf856c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.394823 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f9312b7f-5ea6-4715-a3fb-383ccf856c5b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:33.396041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f9312b7f-5ea6-4715-a3fb-383ccf856c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:33.419107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d07d8-8e16-4fdd-be13-e2491b9ba9e5 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:33.491342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 48/102] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:33.499347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6776ff-6be4-41c0-b33d-51e428869751 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.499820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6776ff-6be4-41c0-b33d-51e428869751 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.500234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6776ff-6be4-41c0-b33d-51e428869751 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.502061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 49/103] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:33.507367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb2f7c-673e-4f1c-9423-fc9129850b80 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.507830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb2f7c-673e-4f1c-9423-fc9129850b80 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.508262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb2f7c-673e-4f1c-9423-fc9129850b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.519543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:33.520612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.520906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.521181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.521969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:33.522287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:33.523748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:33.534630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6bbb2f7c-673e-4f1c-9423-fc9129850b80 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:33.536015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6bbb2f7c-673e-4f1c-9423-fc9129850b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:33.546096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 55/104] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:33.550525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:33.610713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 50/105] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:33.611976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb2f7c-673e-4f1c-9423-fc9129850b80 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:33.711141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb2f7c-673e-4f1c-9423-fc9129850b80 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:33.744842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527be36f-df23-4e29-9503-9cbf01a9dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:33.745903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527be36f-df23-4e29-9503-9cbf01a9dd19 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.746935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527be36f-df23-4e29-9503-9cbf01a9dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.747372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527be36f-df23-4e29-9503-9cbf01a9dd19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.758015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527be36f-df23-4e29-9503-9cbf01a9dd19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-527be36f-df23-4e29-9503-9cbf01a9dd19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:33.766084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:33.767110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:33.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:33.770019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 51/106] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:33.774663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527be36f-df23-4e29-9503-9cbf01a9dd19 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:33.825514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf03430-ccc5-4834-882d-0fee15ab1756 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:33.867109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18cbc4-5a06-41cb-a8a1-d0e55e71a37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:33.868342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18cbc4-5a06-41cb-a8a1-d0e55e71a37e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.868818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18cbc4-5a06-41cb-a8a1-d0e55e71a37e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.869263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18cbc4-5a06-41cb-a8a1-d0e55e71a37e None admin] PATH_INFO: `/v3/users/df2ff43339cf407eb4cebe6484112e01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.870816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18cbc4-5a06-41cb-a8a1-d0e55e71a37e None admin] RBAC: Authorizing `identity:get_user(user_id=df2ff43339cf407eb4cebe6484112e01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:33.872534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18cbc4-5a06-41cb-a8a1-d0e55e71a37e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:33.874951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 56/107] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:33 2022] GET /identity/v3/users/df2ff43339cf407eb4cebe6484112e01 => generated 247 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:33.876179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18cbc4-5a06-41cb-a8a1-d0e55e71a37e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:33.878999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527be36f-df23-4e29-9503-9cbf01a9dd19 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:33.919299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe6ff4-cc57-4b01-bb99-43000cda2ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:33.919740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe6ff4-cc57-4b01-bb99-43000cda2ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.920022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe6ff4-cc57-4b01-bb99-43000cda2ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.920302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe6ff4-cc57-4b01-bb99-43000cda2ce3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.931025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe6ff4-cc57-4b01-bb99-43000cda2ce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2abe6ff4-cc57-4b01-bb99-43000cda2ce3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:33.938981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:33.940391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:33.941285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 52/108] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:33.946995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf1fd-d81c-438c-a0fa-69dcdc832a3a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:33.946995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf1fd-d81c-438c-a0fa-69dcdc832a3a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:33.946995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf1fd-d81c-438c-a0fa-69dcdc832a3a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:33.946995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 53/109] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:33.946995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cf1fd-d81c-438c-a0fa-69dcdc832a3a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:33.996773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18cbc4-5a06-41cb-a8a1-d0e55e71a37e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.029753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.030574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.030763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.031000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.031753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:34.032153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:34.033443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:34.047046 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 57/110] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:34.047948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.056014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cf1fd-d81c-438c-a0fa-69dcdc832a3a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.090783 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ab655-87a6-477c-b078-88d4022da6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.091650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ab655-87a6-477c-b078-88d4022da6dc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.091937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ab655-87a6-477c-b078-88d4022da6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.092216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ab655-87a6-477c-b078-88d4022da6dc None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.093407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ab655-87a6-477c-b078-88d4022da6dc None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:34.094920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ab655-87a6-477c-b078-88d4022da6dc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:34.096752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 54/111] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:33 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:34.097912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56405733-3754-40f9-8daa-93e6e45c2843 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.098205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56405733-3754-40f9-8daa-93e6e45c2843 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.098488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56405733-3754-40f9-8daa-93e6e45c2843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.114577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-56405733-3754-40f9-8daa-93e6e45c2843 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:34.115763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-56405733-3754-40f9-8daa-93e6e45c2843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:34.143015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c4544-dcf3-4639-a02f-00242680ebf4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.175801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b21e05-1340-473e-99c2-1a28dc1d2a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.176527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b21e05-1340-473e-99c2-1a28dc1d2a3c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.176815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b21e05-1340-473e-99c2-1a28dc1d2a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.177091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b21e05-1340-473e-99c2-1a28dc1d2a3c None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.177859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b21e05-1340-473e-99c2-1a28dc1d2a3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:34.178167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b21e05-1340-473e-99c2-1a28dc1d2a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:34.179550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b21e05-1340-473e-99c2-1a28dc1d2a3c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:34.184067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 55/112] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:34.184916 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56405733-3754-40f9-8daa-93e6e45c2843 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.187609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 58/113] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:34.188967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462371a3-1876-4cee-965e-f467b03546fb None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.189256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462371a3-1876-4cee-965e-f467b03546fb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.189532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462371a3-1876-4cee-965e-f467b03546fb None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.190799 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 59/114] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:34.191997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710ed60-cb96-40df-ba9f-ce58cf8f158f None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.192287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710ed60-cb96-40df-ba9f-ce58cf8f158f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.192565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710ed60-cb96-40df-ba9f-ce58cf8f158f None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.193975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 60/115] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:34.194545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710ed60-cb96-40df-ba9f-ce58cf8f158f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.284157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56405733-3754-40f9-8daa-93e6e45c2843 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.289256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710ed60-cb96-40df-ba9f-ce58cf8f158f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.315903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f7ef4-f663-453d-bc83-f6503ab2cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.316782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f7ef4-f663-453d-bc83-f6503ab2cb45 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.317093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f7ef4-f663-453d-bc83-f6503ab2cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.317371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f7ef4-f663-453d-bc83-f6503ab2cb45 None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.318593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f7ef4-f663-453d-bc83-f6503ab2cb45 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:34.320037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f7ef4-f663-453d-bc83-f6503ab2cb45 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:34.321276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480532e-7f04-43b1-8b89-5067dfd3135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.321662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 56/116] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:34 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:34.322822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efaaa6-c2a2-4e1c-8b30-a5a35b6e37f2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.323108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efaaa6-c2a2-4e1c-8b30-a5a35b6e37f2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.323382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efaaa6-c2a2-4e1c-8b30-a5a35b6e37f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.326736 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480532e-7f04-43b1-8b89-5067dfd3135b None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.327107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480532e-7f04-43b1-8b89-5067dfd3135b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.327386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480532e-7f04-43b1-8b89-5067dfd3135b None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01/users/df2ff43339cf407eb4cebe6484112e01/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.329364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480532e-7f04-43b1-8b89-5067dfd3135b None admin] RBAC: Authorizing `identity:create_grant(project_id=1eafe2a57502441faa06c7ab6de43b01, user_id=df2ff43339cf407eb4cebe6484112e01, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:34.330832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480532e-7f04-43b1-8b89-5067dfd3135b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:34.338886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e1efaaa6-c2a2-4e1c-8b30-a5a35b6e37f2 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:34.340601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e1efaaa6-c2a2-4e1c-8b30-a5a35b6e37f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:34.403739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 57/117] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:34.404587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efaaa6-c2a2-4e1c-8b30-a5a35b6e37f2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.411200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 61/118] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:34 2022] PUT /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01/users/df2ff43339cf407eb4cebe6484112e01/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:34.412577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66222ba6-ae7d-45ed-b20a-1521b814527f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.412957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66222ba6-ae7d-45ed-b20a-1521b814527f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.416746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66222ba6-ae7d-45ed-b20a-1521b814527f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.433444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-66222ba6-ae7d-45ed-b20a-1521b814527f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:34.435453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-66222ba6-ae7d-45ed-b20a-1521b814527f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:34.518666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 62/119] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:34.519570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66222ba6-ae7d-45ed-b20a-1521b814527f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.538095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efaaa6-c2a2-4e1c-8b30-a5a35b6e37f2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.574712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b55638-03e7-4114-bbca-cd0cfc365c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.575603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55638-03e7-4114-bbca-cd0cfc365c21 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.575892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55638-03e7-4114-bbca-cd0cfc365c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.576175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b55638-03e7-4114-bbca-cd0cfc365c21 None admin] PATH_INFO: `/v3/users/8061e5277d3743a3bb68b194cf969b27` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.592027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b55638-03e7-4114-bbca-cd0cfc365c21 None admin] RBAC: Authorizing `identity:get_user(user_id=8061e5277d3743a3bb68b194cf969b27)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:34.593564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b55638-03e7-4114-bbca-cd0cfc365c21 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:34.599937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 58/120] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:34 2022] GET /identity/v3/users/8061e5277d3743a3bb68b194cf969b27 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:34.599937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b55638-03e7-4114-bbca-cd0cfc365c21 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.632401 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66222ba6-ae7d-45ed-b20a-1521b814527f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.672918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508af8c-4511-417b-839b-a6daa726c463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.673719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508af8c-4511-417b-839b-a6daa726c463 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.674038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508af8c-4511-417b-839b-a6daa726c463 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.674320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508af8c-4511-417b-839b-a6daa726c463 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.675924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508af8c-4511-417b-839b-a6daa726c463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:34.677770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508af8c-4511-417b-839b-a6daa726c463 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:34.680183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 63/121] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:34.685525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508af8c-4511-417b-839b-a6daa726c463 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.699633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b55638-03e7-4114-bbca-cd0cfc365c21 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.732562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4dcdf-f2b1-4c64-8c55-ed28db6bb54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.733428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4dcdf-f2b1-4c64-8c55-ed28db6bb54f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.733716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4dcdf-f2b1-4c64-8c55-ed28db6bb54f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.734022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4dcdf-f2b1-4c64-8c55-ed28db6bb54f None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.735228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4dcdf-f2b1-4c64-8c55-ed28db6bb54f None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:34.736702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4dcdf-f2b1-4c64-8c55-ed28db6bb54f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:34.742854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 59/122] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:34 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:34.742854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4dcdf-f2b1-4c64-8c55-ed28db6bb54f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.801744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508af8c-4511-417b-839b-a6daa726c463 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.837762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4dcdf-f2b1-4c64-8c55-ed28db6bb54f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.841270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e09adc-1995-4c53-9201-29253c245676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.842117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e09adc-1995-4c53-9201-29253c245676 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.842404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e09adc-1995-4c53-9201-29253c245676 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.842680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e09adc-1995-4c53-9201-29253c245676 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.852801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e09adc-1995-4c53-9201-29253c245676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-e4e09adc-1995-4c53-9201-29253c245676 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:34.861009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:34.863200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 64/123] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:03:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:34.864163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e09adc-1995-4c53-9201-29253c245676 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.877056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fea7a-e42b-4cbb-bd3e-7568326dd454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.877886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fea7a-e42b-4cbb-bd3e-7568326dd454 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.878173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fea7a-e42b-4cbb-bd3e-7568326dd454 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.878448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fea7a-e42b-4cbb-bd3e-7568326dd454 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:34.889750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fea7a-e42b-4cbb-bd3e-7568326dd454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f87fea7a-e42b-4cbb-bd3e-7568326dd454 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:34.898456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:34.899320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:34.900262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 60/124] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:34.901127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fea7a-e42b-4cbb-bd3e-7568326dd454 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:34.964770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e09adc-1995-4c53-9201-29253c245676 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.997056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabf58a-843d-453e-ae32-db35586cdb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:34.997672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fea7a-e42b-4cbb-bd3e-7568326dd454 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:34.998054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabf58a-843d-453e-ae32-db35586cdb84 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:34.998054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabf58a-843d-453e-ae32-db35586cdb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:34.998369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabf58a-843d-453e-ae32-db35586cdb84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.030329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c5ff7-95d5-4b54-a2be-5ede24443f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.030997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5ff7-95d5-4b54-a2be-5ede24443f56 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.031291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5ff7-95d5-4b54-a2be-5ede24443f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.031576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5ff7-95d5-4b54-a2be-5ede24443f56 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.032291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c5ff7-95d5-4b54-a2be-5ede24443f56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.033751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c5ff7-95d5-4b54-a2be-5ede24443f56 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.066057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabf58a-843d-453e-ae32-db35586cdb84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.066057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabf58a-843d-453e-ae32-db35586cdb84 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.121195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 61/125] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:03:34 2022] POST /identity/v3/projects => generated 302 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:35.122073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c5ff7-95d5-4b54-a2be-5ede24443f56 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.166944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 65/126] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:35.168807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabf58a-843d-453e-ae32-db35586cdb84 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.253722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c5ff7-95d5-4b54-a2be-5ede24443f56 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:35.288518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabf58a-843d-453e-ae32-db35586cdb84 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:35.293439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d413bf-730e-4425-af14-2f7a3f941913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.293860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d413bf-730e-4425-af14-2f7a3f941913 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.294153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d413bf-730e-4425-af14-2f7a3f941913 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.294433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d413bf-730e-4425-af14-2f7a3f941913 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01/users/1ff1944c64d841c3a8596ed53714d48d/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.296507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d413bf-730e-4425-af14-2f7a3f941913 None admin] RBAC: Authorizing `identity:create_grant(project_id=1eafe2a57502441faa06c7ab6de43b01, user_id=1ff1944c64d841c3a8596ed53714d48d, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.298001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d413bf-730e-4425-af14-2f7a3f941913 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.330259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.331010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.331423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.331801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.332662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:35.333083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.334630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.344121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 66/127] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:03:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:35.344864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.383370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 62/128] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:35 2022] PUT /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01/users/1ff1944c64d841c3a8596ed53714d48d/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:35.384219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d413bf-730e-4425-af14-2f7a3f941913 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.454139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f242df-e8d0-4cf4-b85b-7ea63a542826 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:35.489947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.490997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.491550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.492038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.493051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:35.493581 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.495283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.505769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 67/129] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:35.507033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.515200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d413bf-730e-4425-af14-2f7a3f941913 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:35.556062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910013af-aeb0-43ac-b852-67e5b1633202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.557021 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910013af-aeb0-43ac-b852-67e5b1633202 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.557224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910013af-aeb0-43ac-b852-67e5b1633202 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.557508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910013af-aeb0-43ac-b852-67e5b1633202 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.559674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910013af-aeb0-43ac-b852-67e5b1633202 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.561156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910013af-aeb0-43ac-b852-67e5b1633202 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.563100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 63/130] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:35 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:35.563824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910013af-aeb0-43ac-b852-67e5b1633202 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.624058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279df5f-c1e1-4a2c-b85a-221107ba6523 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:35.664359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeac275-fd8f-421d-80a5-8667a01c3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.665658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeac275-fd8f-421d-80a5-8667a01c3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.666221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeac275-fd8f-421d-80a5-8667a01c3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.666719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeac275-fd8f-421d-80a5-8667a01c3ec7 None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.668887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeac275-fd8f-421d-80a5-8667a01c3ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.670680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeac275-fd8f-421d-80a5-8667a01c3ec7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.671038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910013af-aeb0-43ac-b852-67e5b1633202 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:35.672839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 68/131] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:35 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:35.678151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeac275-fd8f-421d-80a5-8667a01c3ec7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.707674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f572a-0f95-4a9f-b897-f3aac01f484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.708612 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f572a-0f95-4a9f-b897-f3aac01f484d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.708907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f572a-0f95-4a9f-b897-f3aac01f484d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.709190 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f572a-0f95-4a9f-b897-f3aac01f484d None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.710713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f572a-0f95-4a9f-b897-f3aac01f484d None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.712171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f572a-0f95-4a9f-b897-f3aac01f484d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.714119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 64/132] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:35 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:35.715309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f572a-0f95-4a9f-b897-f3aac01f484d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.785800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeac275-fd8f-421d-80a5-8667a01c3ec7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:35.819669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f572a-0f95-4a9f-b897-f3aac01f484d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:35.821655 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.822597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.828204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.828204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.828204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] RBAC: Adding query filter params (role.id=1163600e955a4a41905c8cc295ce219b, scope.project.id=1eafe2a57502441faa06c7ab6de43b01, user.id=8061e5277d3743a3bb68b194cf969b27) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:35.828204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.828204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.832563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 69/133] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:35 2022] GET /identity/v3/role_assignments?user.id=8061e5277d3743a3bb68b194cf969b27&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=1163600e955a4a41905c8cc295ce219b => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:35.833705 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:35.863912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee82471-3396-4acb-835a-0392d9fcb2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:35.868211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee82471-3396-4acb-835a-0392d9fcb2bd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:35.868211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee82471-3396-4acb-835a-0392d9fcb2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:35.868211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee82471-3396-4acb-835a-0392d9fcb2bd None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:35.869672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee82471-3396-4acb-835a-0392d9fcb2bd None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:35.871224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee82471-3396-4acb-835a-0392d9fcb2bd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:35.873149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 65/134] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:35 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:35.881889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee82471-3396-4acb-835a-0392d9fcb2bd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:36.005616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a827-2073-4421-9a80-0aa4da38a37d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:36.070009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee82471-3396-4acb-835a-0392d9fcb2bd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:36.074622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633003d-1b84-43bb-a84b-a57a61e6cbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:36.075519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633003d-1b84-43bb-a84b-a57a61e6cbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:36.075808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633003d-1b84-43bb-a84b-a57a61e6cbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:36.076091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633003d-1b84-43bb-a84b-a57a61e6cbf5 None admin] PATH_INFO: `/v3/projects/e54f643f23454f1d954d6649c0103db4` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:36.081780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633003d-1b84-43bb-a84b-a57a61e6cbf5 None admin] RBAC: Authorizing `identity:get_project(project_id=e54f643f23454f1d954d6649c0103db4)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:36.083354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633003d-1b84-43bb-a84b-a57a61e6cbf5 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:36.085482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 70/135] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:35 2022] GET /identity/v3/projects/e54f643f23454f1d954d6649c0103db4 => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:36.091307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633003d-1b84-43bb-a84b-a57a61e6cbf5 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:36.145406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ef887-6247-4935-8446-d1c736d8575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:36.146328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ef887-6247-4935-8446-d1c736d8575d None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:36.146661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ef887-6247-4935-8446-d1c736d8575d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:36.146951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ef887-6247-4935-8446-d1c736d8575d None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01/users/1ff1944c64d841c3a8596ed53714d48d/roles/9065c36fc305418baa5b977858b445dd` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:36.152433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ef887-6247-4935-8446-d1c736d8575d None admin] RBAC: Authorizing `identity:create_grant(project_id=1eafe2a57502441faa06c7ab6de43b01, user_id=1ff1944c64d841c3a8596ed53714d48d, role_id=9065c36fc305418baa5b977858b445dd)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:36.156024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ef887-6247-4935-8446-d1c736d8575d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:36.242346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 66/136] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:35 2022] PUT /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01/users/1ff1944c64d841c3a8596ed53714d48d/roles/9065c36fc305418baa5b977858b445dd => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:36.243056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633003d-1b84-43bb-a84b-a57a61e6cbf5 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:36.291238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaa7cb-441a-4269-8baa-9c6e93413f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:36.292638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaa7cb-441a-4269-8baa-9c6e93413f8a None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:36.293680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaa7cb-441a-4269-8baa-9c6e93413f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:36.298352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaa7cb-441a-4269-8baa-9c6e93413f8a None admin] PATH_INFO: `/v3/projects/e54f643f23454f1d954d6649c0103db4/users/1ff1944c64d841c3a8596ed53714d48d/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:36.300725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aaa7cb-441a-4269-8baa-9c6e93413f8a None admin] RBAC: Authorizing `identity:create_grant(project_id=e54f643f23454f1d954d6649c0103db4, user_id=1ff1944c64d841c3a8596ed53714d48d, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:36.302341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aaa7cb-441a-4269-8baa-9c6e93413f8a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:36.408142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 71/137] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:36 2022] PUT /identity/v3/projects/e54f643f23454f1d954d6649c0103db4/users/1ff1944c64d841c3a8596ed53714d48d/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:37.163441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd94707-1c47-4119-b8b3-0e9162d4ef85 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:37.163952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd94707-1c47-4119-b8b3-0e9162d4ef85 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:37.164055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd94707-1c47-4119-b8b3-0e9162d4ef85 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:37.165605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 67/138] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:37.190840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480eb66-a696-4556-b273-4d4b2d78f6a5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:37.191347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480eb66-a696-4556-b273-4d4b2d78f6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:37.191421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480eb66-a696-4556-b273-4d4b2d78f6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:37.209278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-3480eb66-a696-4556-b273-4d4b2d78f6a5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:37.210531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3480eb66-a696-4556-b273-4d4b2d78f6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:37.286349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 72/139] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:37.320853 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd94707-1c47-4119-b8b3-0e9162d4ef85 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:37.445230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd94707-1c47-4119-b8b3-0e9162d4ef85 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:37.487428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8758890-2916-458a-a22b-871501efa1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:37.488237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758890-2916-458a-a22b-871501efa1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:37.488527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758890-2916-458a-a22b-871501efa1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:37.488811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758890-2916-458a-a22b-871501efa1e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:37.506725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8758890-2916-458a-a22b-871501efa1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c8758890-2916-458a-a22b-871501efa1e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:37.514671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:37.515515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:37.516278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 68/140] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:37.521885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480eb66-a696-4556-b273-4d4b2d78f6a5 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:37.649565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480eb66-a696-4556-b273-4d4b2d78f6a5 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:37.693952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a523887-c1a0-4457-8250-26fe6101ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:37.694995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a523887-c1a0-4457-8250-26fe6101ba67 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:37.695529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a523887-c1a0-4457-8250-26fe6101ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:37.695998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a523887-c1a0-4457-8250-26fe6101ba67 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:37.696937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a523887-c1a0-4457-8250-26fe6101ba67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:37.697453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a523887-c1a0-4457-8250-26fe6101ba67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:37.699088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a523887-c1a0-4457-8250-26fe6101ba67 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:37.712861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 73/141] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:37.718493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8758890-2916-458a-a22b-871501efa1e6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:37.842844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb051c-2ab7-4e1d-8b38-3bd224a0885a None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:37.843521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb051c-2ab7-4e1d-8b38-3bd224a0885a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:37.843998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb051c-2ab7-4e1d-8b38-3bd224a0885a None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:37.845559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 74/142] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:37.850887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d801-7914-4c65-8db6-0dde0265aaaa None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:37.851404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d801-7914-4c65-8db6-0dde0265aaaa None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:37.851878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d801-7914-4c65-8db6-0dde0265aaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:37.877811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8758890-2916-458a-a22b-871501efa1e6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:37.885220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-51d4d801-7914-4c65-8db6-0dde0265aaaa None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:37.887239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-51d4d801-7914-4c65-8db6-0dde0265aaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:37.931914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45171c2f-1ca4-4cfe-9ebd-399d95381451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:37.933140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45171c2f-1ca4-4cfe-9ebd-399d95381451 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:37.934005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45171c2f-1ca4-4cfe-9ebd-399d95381451 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:37.934299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45171c2f-1ca4-4cfe-9ebd-399d95381451 None admin] PATH_INFO: `/v3/users/df2ff43339cf407eb4cebe6484112e01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:37.935549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45171c2f-1ca4-4cfe-9ebd-399d95381451 None admin] RBAC: Authorizing `identity:get_user(user_id=df2ff43339cf407eb4cebe6484112e01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:37.937017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45171c2f-1ca4-4cfe-9ebd-399d95381451 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:37.938694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 69/143] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:37 2022] GET /identity/v3/users/df2ff43339cf407eb4cebe6484112e01 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:37.944100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45171c2f-1ca4-4cfe-9ebd-399d95381451 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:38.048025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 75/144] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:38.085305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4d801-7914-4c65-8db6-0dde0265aaaa None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:38.155711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45171c2f-1ca4-4cfe-9ebd-399d95381451 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:38.234944 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11597a-016f-4d85-a3ed-ceba0eec2224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:38.235696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11597a-016f-4d85-a3ed-ceba0eec2224 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:38.235901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11597a-016f-4d85-a3ed-ceba0eec2224 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:38.236188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11597a-016f-4d85-a3ed-ceba0eec2224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:38.314568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4d801-7914-4c65-8db6-0dde0265aaaa None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:38.381975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543b7a-9082-4d7d-990f-f856c9eef103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:38.382798 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543b7a-9082-4d7d-990f-f856c9eef103 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:38.383092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543b7a-9082-4d7d-990f-f856c9eef103 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:38.383376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543b7a-9082-4d7d-990f-f856c9eef103 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:38.384590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd543b7a-9082-4d7d-990f-f856c9eef103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:38.386136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd543b7a-9082-4d7d-990f-f856c9eef103 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:38.387900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 76/145] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:38.396669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543b7a-9082-4d7d-990f-f856c9eef103 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:38.398020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11597a-016f-4d85-a3ed-ceba0eec2224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:38.399743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11597a-016f-4d85-a3ed-ceba0eec2224 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:38.556135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543b7a-9082-4d7d-990f-f856c9eef103 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:38.557544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 70/146] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:38.559573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11597a-016f-4d85-a3ed-ceba0eec2224 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:38.620521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829e35e-6858-46a3-ac2a-c1879d55e5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:38.621673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829e35e-6858-46a3-ac2a-c1879d55e5ee None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:38.622223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829e35e-6858-46a3-ac2a-c1879d55e5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:38.622707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829e35e-6858-46a3-ac2a-c1879d55e5ee None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:38.623934 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829e35e-6858-46a3-ac2a-c1879d55e5ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:38.625455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829e35e-6858-46a3-ac2a-c1879d55e5ee None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:38.710586 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11597a-016f-4d85-a3ed-ceba0eec2224 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:38.750497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9dda9-5445-4f80-b343-b853faa170b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:38.751371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9dda9-5445-4f80-b343-b853faa170b2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:38.751658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9dda9-5445-4f80-b343-b853faa170b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:38.751935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9dda9-5445-4f80-b343-b853faa170b2 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:38.753198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9dda9-5445-4f80-b343-b853faa170b2 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:38.754693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9dda9-5445-4f80-b343-b853faa170b2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:38.756401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 71/147] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:38 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:38.762182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9dda9-5445-4f80-b343-b853faa170b2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:38.770541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 77/148] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:03:38 2022] POST /identity/v3/users => generated 283 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:38.872119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9dda9-5445-4f80-b343-b853faa170b2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:38.908886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa2487-f450-4e8a-b832-b386155e8a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:38.909568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa2487-f450-4e8a-b832-b386155e8a71 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:38.909901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa2487-f450-4e8a-b832-b386155e8a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:38.910188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa2487-f450-4e8a-b832-b386155e8a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:38.911041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa2487-f450-4e8a-b832-b386155e8a71 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=1eafe2a57502441faa06c7ab6de43b01, user.id=df2ff43339cf407eb4cebe6484112e01) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:38.911357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa2487-f450-4e8a-b832-b386155e8a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:38.912816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa2487-f450-4e8a-b832-b386155e8a71 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:38.919795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 72/149] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:38 2022] GET /identity/v3/role_assignments?user.id=df2ff43339cf407eb4cebe6484112e01&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=663341777b0b44128ed78fe8d40f6462 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:39.202208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049d1e-2783-4e53-ad29-0db617de47df None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.202426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049d1e-2783-4e53-ad29-0db617de47df None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.202716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049d1e-2783-4e53-ad29-0db617de47df None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.204214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 78/150] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:39.207950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf311b4-a80c-4b86-8590-df34cff0da01 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.208249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf311b4-a80c-4b86-8590-df34cff0da01 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.208536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf311b4-a80c-4b86-8590-df34cff0da01 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.210297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec436-c1cb-4926-a7e8-d3a0c6db77af None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.210493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 73/151] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:39.210585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec436-c1cb-4926-a7e8-d3a0c6db77af None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.210865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec436-c1cb-4926-a7e8-d3a0c6db77af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.219990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32053403-b052-4db4-be62-fd23896d2cd1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.220288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32053403-b052-4db4-be62-fd23896d2cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.220565 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32053403-b052-4db4-be62-fd23896d2cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.229304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d63ec436-c1cb-4926-a7e8-d3a0c6db77af None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:39.231096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d63ec436-c1cb-4926-a7e8-d3a0c6db77af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:39.241344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-32053403-b052-4db4-be62-fd23896d2cd1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:39.242578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-32053403-b052-4db4-be62-fd23896d2cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:39.315855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 79/152] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:39.329414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 74/153] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:39.343653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ec436-c1cb-4926-a7e8-d3a0c6db77af None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:39.356308 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32053403-b052-4db4-be62-fd23896d2cd1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:39.458941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32053403-b052-4db4-be62-fd23896d2cd1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:39.466778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ec436-c1cb-4926-a7e8-d3a0c6db77af None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:39.502048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27992d96-7015-40b2-ac4d-2cdcd3cbab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:39.503256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992d96-7015-40b2-ac4d-2cdcd3cbab2f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.503759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992d96-7015-40b2-ac4d-2cdcd3cbab2f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.504232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992d96-7015-40b2-ac4d-2cdcd3cbab2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.509723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac60587-f037-4f95-894c-4065d97cc536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:39.510767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac60587-f037-4f95-894c-4065d97cc536 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.511225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac60587-f037-4f95-894c-4065d97cc536 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.511681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac60587-f037-4f95-894c-4065d97cc536 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.519669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27992d96-7015-40b2-ac4d-2cdcd3cbab2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-27992d96-7015-40b2-ac4d-2cdcd3cbab2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:39.529525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:39.530529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:39.531320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:39.532917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 75/154] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:39.534714 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac60587-f037-4f95-894c-4065d97cc536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:39.538839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27992d96-7015-40b2-ac4d-2cdcd3cbab2f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-2ac60587-f037-4f95-894c-4065d97cc536 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:39.546242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:39.547199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:39.547969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 80/155] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:39.554456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac60587-f037-4f95-894c-4065d97cc536 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:39.667033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac60587-f037-4f95-894c-4065d97cc536 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:39.700453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27992d96-7015-40b2-ac4d-2cdcd3cbab2f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:39.708307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:39.709038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.709263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.709539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.710349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:39.710654 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:39.712055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:39.720832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 81/156] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:39.726236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:39.740293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:39.741030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.741322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.741602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.742495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:39.742811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:39.745270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:39.765471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 76/157] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:39.768649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:39.885364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cec297-6788-4c66-be72-4a7a1a345efd None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:39.893372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa3899-1bf2-4125-9ae6-a4ec95985ae0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:39.925932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5958c-f32a-4bbc-b169-f78699c356aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:39.926948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5958c-f32a-4bbc-b169-f78699c356aa None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.927380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5958c-f32a-4bbc-b169-f78699c356aa None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.927936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5958c-f32a-4bbc-b169-f78699c356aa None admin] PATH_INFO: `/v3/users/8061e5277d3743a3bb68b194cf969b27` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.929425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5958c-f32a-4bbc-b169-f78699c356aa None admin] RBAC: Authorizing `identity:get_user(user_id=8061e5277d3743a3bb68b194cf969b27)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:39.930918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5958c-f32a-4bbc-b169-f78699c356aa None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:39.932979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 82/158] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:39 2022] GET /identity/v3/users/8061e5277d3743a3bb68b194cf969b27 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:39.933089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcd19a-8f87-4906-a229-fafed55ba4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:39.933988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcd19a-8f87-4906-a229-fafed55ba4bc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.934099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f261372-fd2b-4d6b-9009-119c40462666 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.934275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcd19a-8f87-4906-a229-fafed55ba4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.934372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f261372-fd2b-4d6b-9009-119c40462666 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.934551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcd19a-8f87-4906-a229-fafed55ba4bc None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.934662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f261372-fd2b-4d6b-9009-119c40462666 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.935979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcd19a-8f87-4906-a229-fafed55ba4bc None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:39.936153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 83/159] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:39.937425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcd19a-8f87-4906-a229-fafed55ba4bc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:39.939335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 77/160] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:39 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:39.940552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f261372-fd2b-4d6b-9009-119c40462666 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:39.944959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42ce8c-0a23-40e5-931e-e19c474c23b5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:39.945258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42ce8c-0a23-40e5-931e-e19c474c23b5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:39.945542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42ce8c-0a23-40e5-931e-e19c474c23b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:39.967415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3f42ce8c-0a23-40e5-931e-e19c474c23b5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:39.968547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3f42ce8c-0a23-40e5-931e-e19c474c23b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:40.047539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 78/161] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:40.048724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42ce8c-0a23-40e5-931e-e19c474c23b5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:40.065419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f261372-fd2b-4d6b-9009-119c40462666 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:40.114636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c87a67-1678-48a7-a2a1-2dba114f8478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:40.116375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c87a67-1678-48a7-a2a1-2dba114f8478 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.116810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c87a67-1678-48a7-a2a1-2dba114f8478 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.117416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c87a67-1678-48a7-a2a1-2dba114f8478 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.119395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c87a67-1678-48a7-a2a1-2dba114f8478 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:40.121063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c87a67-1678-48a7-a2a1-2dba114f8478 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:40.123875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 84/162] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:39 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:40.125342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db2865-f308-44b0-ae9a-495efb1ea65c None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.125885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db2865-f308-44b0-ae9a-495efb1ea65c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.126452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db2865-f308-44b0-ae9a-495efb1ea65c None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.127883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 85/163] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:40.129554 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db2865-f308-44b0-ae9a-495efb1ea65c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:40.160920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42ce8c-0a23-40e5-931e-e19c474c23b5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:40.201298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f427df-103e-4f9c-8bed-df9d5066f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:40.201991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f427df-103e-4f9c-8bed-df9d5066f5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.202279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f427df-103e-4f9c-8bed-df9d5066f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.202561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f427df-103e-4f9c-8bed-df9d5066f5a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.253774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db2865-f308-44b0-ae9a-495efb1ea65c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:40.283855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f427df-103e-4f9c-8bed-df9d5066f5a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:40.286786 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f427df-103e-4f9c-8bed-df9d5066f5a5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:40.290184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29876bd0-3f7f-4dea-a21e-1b0722764241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:40.291238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876bd0-3f7f-4dea-a21e-1b0722764241 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.291724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876bd0-3f7f-4dea-a21e-1b0722764241 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.292216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876bd0-3f7f-4dea-a21e-1b0722764241 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.305644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29876bd0-3f7f-4dea-a21e-1b0722764241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-29876bd0-3f7f-4dea-a21e-1b0722764241 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:40.315109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:40.316420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:40.317271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:40.319965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 86/164] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:40.321472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42969a-b355-4e09-99e1-ad7ab2dd724a None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.325642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42969a-b355-4e09-99e1-ad7ab2dd724a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.326233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42969a-b355-4e09-99e1-ad7ab2dd724a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.348001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-dc42969a-b355-4e09-99e1-ad7ab2dd724a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:40.349521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-dc42969a-b355-4e09-99e1-ad7ab2dd724a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:40.404665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 79/165] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:40.410158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f427df-103e-4f9c-8bed-df9d5066f5a5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:40.422445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 87/166] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:40.423661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42969a-b355-4e09-99e1-ad7ab2dd724a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:40.531017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f427df-103e-4f9c-8bed-df9d5066f5a5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:40.535534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42969a-b355-4e09-99e1-ad7ab2dd724a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:40.567805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2db5e-9151-4d15-a6d2-77a310213df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:40.569086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2db5e-9151-4d15-a6d2-77a310213df9 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.569569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2db5e-9151-4d15-a6d2-77a310213df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.570079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2db5e-9151-4d15-a6d2-77a310213df9 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01/users/8061e5277d3743a3bb68b194cf969b27/roles/1163600e955a4a41905c8cc295ce219b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.572412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2db5e-9151-4d15-a6d2-77a310213df9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1eafe2a57502441faa06c7ab6de43b01, user_id=8061e5277d3743a3bb68b194cf969b27, role_id=1163600e955a4a41905c8cc295ce219b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:40.574984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2db5e-9151-4d15-a6d2-77a310213df9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:40.576527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:40.577199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.577487 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.577768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.578722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:40.579036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:40.581001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:40.589354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 88/167] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:40.590520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:40.669460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 80/168] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:40 2022] PUT /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01/users/8061e5277d3743a3bb68b194cf969b27/roles/1163600e955a4a41905c8cc295ce219b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:40.670652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2db5e-9151-4d15-a6d2-77a310213df9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:40.703805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622dd0c-dbf1-468f-b9b5-4dafdf01b355 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:40.757170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec815d5-14b1-4fc1-a405-519cc29d986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:40.758083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec815d5-14b1-4fc1-a405-519cc29d986c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.758374 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec815d5-14b1-4fc1-a405-519cc29d986c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.758649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec815d5-14b1-4fc1-a405-519cc29d986c None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.765810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec815d5-14b1-4fc1-a405-519cc29d986c None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:40.765810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec815d5-14b1-4fc1-a405-519cc29d986c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:40.766211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 89/169] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:40 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:40.767126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec815d5-14b1-4fc1-a405-519cc29d986c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:40.889610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2db5e-9151-4d15-a6d2-77a310213df9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:40.925724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd320dc4-eb2a-41ba-8846-ab79f0d834a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:40.926938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd320dc4-eb2a-41ba-8846-ab79f0d834a5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.927443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd320dc4-eb2a-41ba-8846-ab79f0d834a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.927914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd320dc4-eb2a-41ba-8846-ab79f0d834a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.929252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec815d5-14b1-4fc1-a405-519cc29d986c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:40.940160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd320dc4-eb2a-41ba-8846-ab79f0d834a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-bd320dc4-eb2a-41ba-8846-ab79f0d834a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:40.948778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:40.949849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:40.950635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:40.953190 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 81/170] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:03:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:40.954288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd320dc4-eb2a-41ba-8846-ab79f0d834a5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:40.993105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0ef85-9976-4909-83a2-b377e633b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:40.994042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0ef85-9976-4909-83a2-b377e633b251 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:40.994333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0ef85-9976-4909-83a2-b377e633b251 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:40.994617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0ef85-9976-4909-83a2-b377e633b251 None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:40.998963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0ef85-9976-4909-83a2-b377e633b251 None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.000612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0ef85-9976-4909-83a2-b377e633b251 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.002660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 90/171] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:40 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:41.003251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0ef85-9976-4909-83a2-b377e633b251 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:41.086934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd320dc4-eb2a-41ba-8846-ab79f0d834a5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:41.123058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:41.124039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0ef85-9976-4909-83a2-b377e633b251 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:41.124357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:41.124838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:41.125316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:41.126438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] RBAC: Adding query filter params (role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=1eafe2a57502441faa06c7ab6de43b01, user.id=1ff1944c64d841c3a8596ed53714d48d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:41.126983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.128663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.135332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 82/172] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:40 2022] GET /identity/v3/role_assignments?user.id=1ff1944c64d841c3a8596ed53714d48d&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=3552540dc0054d2e87fad160a358c58b => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:41.136477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:41.183016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:41.184173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:41.184461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:41.184738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:41.185509 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:41.186055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.187246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.196128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 91/173] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:03:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:41.202330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:41.252416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bda33-f225-4a28-bea3-0dc710ac647d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:41.288073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f807680-ba08-4da5-881a-d782fb4e45a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:41.289495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807680-ba08-4da5-881a-d782fb4e45a1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:41.290182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807680-ba08-4da5-881a-d782fb4e45a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:41.290661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807680-ba08-4da5-881a-d782fb4e45a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:41.340455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fc560-8aaf-48a9-a068-1be3ac5c60fb None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:41.377218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f807680-ba08-4da5-881a-d782fb4e45a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.379039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f807680-ba08-4da5-881a-d782fb4e45a1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.394699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276d32a-2f2f-4521-8097-2d6781305b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:41.395587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276d32a-2f2f-4521-8097-2d6781305b8d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:41.395878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276d32a-2f2f-4521-8097-2d6781305b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:41.396155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276d32a-2f2f-4521-8097-2d6781305b8d None admin] PATH_INFO: `/v3/users/1ff1944c64d841c3a8596ed53714d48d` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:41.397464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276d32a-2f2f-4521-8097-2d6781305b8d None admin] RBAC: Authorizing `identity:get_user(user_id=1ff1944c64d841c3a8596ed53714d48d)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.398972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276d32a-2f2f-4521-8097-2d6781305b8d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.400790 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 92/174] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:41 2022] GET /identity/v3/users/1ff1944c64d841c3a8596ed53714d48d => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:41.406889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276d32a-2f2f-4521-8097-2d6781305b8d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:41.485024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 83/175] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:41.490673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f807680-ba08-4da5-881a-d782fb4e45a1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:41.547382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276d32a-2f2f-4521-8097-2d6781305b8d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:41.586376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b9f84-f02a-44a5-b703-46ed91d8477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:41.587034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b9f84-f02a-44a5-b703-46ed91d8477d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:41.587322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b9f84-f02a-44a5-b703-46ed91d8477d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:41.587603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b9f84-f02a-44a5-b703-46ed91d8477d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:41.594973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f807680-ba08-4da5-881a-d782fb4e45a1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:41.631266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5224bd-d3fd-409b-8dc0-cc8189d474bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:41.632251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5224bd-d3fd-409b-8dc0-cc8189d474bf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:41.632603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5224bd-d3fd-409b-8dc0-cc8189d474bf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:41.632953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5224bd-d3fd-409b-8dc0-cc8189d474bf None admin] PATH_INFO: `/v3/projects/e54f643f23454f1d954d6649c0103db4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:41.634323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5224bd-d3fd-409b-8dc0-cc8189d474bf None admin] RBAC: Authorizing `identity:get_project(project_id=e54f643f23454f1d954d6649c0103db4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.635882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5224bd-d3fd-409b-8dc0-cc8189d474bf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.638354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 84/176] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:41 2022] GET /identity/v3/projects/e54f643f23454f1d954d6649c0103db4 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:41.644310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5224bd-d3fd-409b-8dc0-cc8189d474bf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:41.659542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b9f84-f02a-44a5-b703-46ed91d8477d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.661087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b9f84-f02a-44a5-b703-46ed91d8477d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.746810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5224bd-d3fd-409b-8dc0-cc8189d474bf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:41.782455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:41.783119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:41.783402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:41.783680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:41.784522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] RBAC: Adding query filter params (role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=e54f643f23454f1d954d6649c0103db4, user.id=1ff1944c64d841c3a8596ed53714d48d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:41.784837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.786325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.792694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 85/177] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:41 2022] GET /identity/v3/role_assignments?user.id=1ff1944c64d841c3a8596ed53714d48d&scope.project.id=e54f643f23454f1d954d6649c0103db4&role.id=3552540dc0054d2e87fad160a358c58b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:41.804833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 93/178] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:41.811743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:41.944401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175e8dd-88c6-488e-942d-bc1495a27358 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:41.983366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6105e4-1959-44b0-90bf-a49d800247e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:41.984242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6105e4-1959-44b0-90bf-a49d800247e9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:41.984528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6105e4-1959-44b0-90bf-a49d800247e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:41.984810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6105e4-1959-44b0-90bf-a49d800247e9 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:41.986478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6105e4-1959-44b0-90bf-a49d800247e9 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:41.987931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6105e4-1959-44b0-90bf-a49d800247e9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:41.993487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 86/179] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:41 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:41.996186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b9f84-f02a-44a5-b703-46ed91d8477d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:42.122134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b9f84-f02a-44a5-b703-46ed91d8477d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:42.156692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb27fab-3f18-4592-bc57-b47960ac4e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:42.157360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27fab-3f18-4592-bc57-b47960ac4e92 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:42.157645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27fab-3f18-4592-bc57-b47960ac4e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:42.158023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27fab-3f18-4592-bc57-b47960ac4e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:42.159051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27fab-3f18-4592-bc57-b47960ac4e92 None admin] RBAC: Adding query filter params (role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=1eafe2a57502441faa06c7ab6de43b01, user.id=1ff1944c64d841c3a8596ed53714d48d) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:42.159370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27fab-3f18-4592-bc57-b47960ac4e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:42.160832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27fab-3f18-4592-bc57-b47960ac4e92 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:42.169018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 94/180] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:41 2022] GET /identity/v3/role_assignments?user.id=1ff1944c64d841c3a8596ed53714d48d&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=9065c36fc305418baa5b977858b445dd => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:42.781911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae599d-9ad1-4e3f-b6ec-5669f647ba58 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:42.782410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae599d-9ad1-4e3f-b6ec-5669f647ba58 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:42.782410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae599d-9ad1-4e3f-b6ec-5669f647ba58 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:42.784033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 87/181] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:42.789166 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0745b2f-d411-432f-ac7e-3ec3ad351c20 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:42.790032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0745b2f-d411-432f-ac7e-3ec3ad351c20 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:42.790455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0745b2f-d411-432f-ac7e-3ec3ad351c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:42.807625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f0745b2f-d411-432f-ac7e-3ec3ad351c20 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:42.809121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f0745b2f-d411-432f-ac7e-3ec3ad351c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:42.878328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 95/182] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:42.902772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae599d-9ad1-4e3f-b6ec-5669f647ba58 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.012123 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae599d-9ad1-4e3f-b6ec-5669f647ba58 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.046419 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb33a7-16d4-48f6-b577-278058c80b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:43.047550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb33a7-16d4-48f6-b577-278058c80b7a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.048030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb33a7-16d4-48f6-b577-278058c80b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.048515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb33a7-16d4-48f6-b577-278058c80b7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.059482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb33a7-16d4-48f6-b577-278058c80b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-28fb33a7-16d4-48f6-b577-278058c80b7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:43.067474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:43.068356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:43.069130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:43.071906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 88/183] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:43.078620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0745b2f-d411-432f-ac7e-3ec3ad351c20 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.086571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a24ad-1d42-453e-906a-932f6454e34c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.086913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a24ad-1d42-453e-906a-932f6454e34c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.087209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a24ad-1d42-453e-906a-932f6454e34c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.089038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 89/184] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:43.096365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4cf1c-8458-4c91-86e7-6eb736e0959e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.096659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4cf1c-8458-4c91-86e7-6eb736e0959e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.096946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4cf1c-8458-4c91-86e7-6eb736e0959e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.118362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a2b4cf1c-8458-4c91-86e7-6eb736e0959e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:43.119564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a2b4cf1c-8458-4c91-86e7-6eb736e0959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:43.185502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 90/185] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:43.186656 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286b062-8e99-4408-bdc4-f70a518771b2 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.186945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286b062-8e99-4408-bdc4-f70a518771b2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.187223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286b062-8e99-4408-bdc4-f70a518771b2 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.188623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 91/186] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:43.191193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0745b2f-d411-432f-ac7e-3ec3ad351c20 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.193717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb560e-4375-49c4-a6d9-82fcabf2b095 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.194047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb560e-4375-49c4-a6d9-82fcabf2b095 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.194336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb560e-4375-49c4-a6d9-82fcabf2b095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.216859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bdcb560e-4375-49c4-a6d9-82fcabf2b095 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:43.218849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bdcb560e-4375-49c4-a6d9-82fcabf2b095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:43.227662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:43.228334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.228616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.228891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.229632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:43.229970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:43.231372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:43.243732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 96/187] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:43.243732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.289965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 92/188] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:43.290985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb560e-4375-49c4-a6d9-82fcabf2b095 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.353914 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81b19-f426-41c9-b64c-07a1a7ca1545 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.392262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee16f2-a005-441d-9c73-fac68a9a7bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:43.393081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee16f2-a005-441d-9c73-fac68a9a7bad None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.393365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee16f2-a005-441d-9c73-fac68a9a7bad None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.393640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee16f2-a005-441d-9c73-fac68a9a7bad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.406108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee16f2-a005-441d-9c73-fac68a9a7bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:43.406324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb560e-4375-49c4-a6d9-82fcabf2b095 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-92ee16f2-a005-441d-9c73-fac68a9a7bad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:43.414380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:43.415288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:43.416100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 97/189] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:03:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:43.419373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee16f2-a005-441d-9c73-fac68a9a7bad None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.442161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0749d-079d-413e-a3a0-9826312ad9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:43.443042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0749d-079d-413e-a3a0-9826312ad9aa None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.443328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0749d-079d-413e-a3a0-9826312ad9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.443605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0749d-079d-413e-a3a0-9826312ad9aa None admin] PATH_INFO: `/v3/users/49467f558a2340839aa7c7847f1a5d79` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.459970 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0749d-079d-413e-a3a0-9826312ad9aa None admin] RBAC: Authorizing `identity:get_user(user_id=49467f558a2340839aa7c7847f1a5d79)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:43.461848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0749d-079d-413e-a3a0-9826312ad9aa None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:43.463713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 93/190] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:43 2022] GET /identity/v3/users/49467f558a2340839aa7c7847f1a5d79 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:43.464315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0749d-079d-413e-a3a0-9826312ad9aa None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.523203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee16f2-a005-441d-9c73-fac68a9a7bad None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.556881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f606b9-2f46-4ded-96b7-7980f7541768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:43.558212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f606b9-2f46-4ded-96b7-7980f7541768 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.559051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f606b9-2f46-4ded-96b7-7980f7541768 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.559503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f606b9-2f46-4ded-96b7-7980f7541768 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.561418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f606b9-2f46-4ded-96b7-7980f7541768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:43.563614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f606b9-2f46-4ded-96b7-7980f7541768 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:43.569751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 98/191] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:43.569909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0749d-079d-413e-a3a0-9826312ad9aa None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.570884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f606b9-2f46-4ded-96b7-7980f7541768 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.604781 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:43.606068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.606346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.606624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.607404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:43.607707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:43.609102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:43.617029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 94/192] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:03:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:43.617640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.676822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f606b9-2f46-4ded-96b7-7980f7541768 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.709734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a81eb4-68f1-4d24-9bad-11a6d36719d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:43.710424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a81eb4-68f1-4d24-9bad-11a6d36719d6 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.710711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a81eb4-68f1-4d24-9bad-11a6d36719d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.710992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a81eb4-68f1-4d24-9bad-11a6d36719d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.718475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46bdff-1494-48db-9cb8-b261047fd721 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.751642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72bdaf-ab78-4dfb-b4e3-d71ecbac3726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:43.752317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72bdaf-ab78-4dfb-b4e3-d71ecbac3726 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:43.752602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72bdaf-ab78-4dfb-b4e3-d71ecbac3726 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:43.752882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72bdaf-ab78-4dfb-b4e3-d71ecbac3726 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:43.753588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72bdaf-ab78-4dfb-b4e3-d71ecbac3726 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:43.755095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72bdaf-ab78-4dfb-b4e3-d71ecbac3726 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:43.779566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a81eb4-68f1-4d24-9bad-11a6d36719d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:43.781051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a81eb4-68f1-4d24-9bad-11a6d36719d6 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:43.853482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 95/193] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:03:43 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:43.863022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72bdaf-ab78-4dfb-b4e3-d71ecbac3726 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.883047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 99/194] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:43.889169 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a81eb4-68f1-4d24-9bad-11a6d36719d6 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:43.983265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72bdaf-ab78-4dfb-b4e3-d71ecbac3726 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:43.996544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a81eb4-68f1-4d24-9bad-11a6d36719d6 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:44.022526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9f7c0-bf73-4ec8-aa3a-478b6b5da4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.023697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9f7c0-bf73-4ec8-aa3a-478b6b5da4de None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.023978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9f7c0-bf73-4ec8-aa3a-478b6b5da4de None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.024260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9f7c0-bf73-4ec8-aa3a-478b6b5da4de None admin] PATH_INFO: `/v3/users/49467f558a2340839aa7c7847f1a5d79` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.025526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9f7c0-bf73-4ec8-aa3a-478b6b5da4de None admin] RBAC: Authorizing `identity:get_user(user_id=49467f558a2340839aa7c7847f1a5d79)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:44.027599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9f7c0-bf73-4ec8-aa3a-478b6b5da4de None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:44.033585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 96/195] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:43 2022] GET /identity/v3/users/49467f558a2340839aa7c7847f1a5d79 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:44.033585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a2cb2-0804-4821-b19f-8f24190cfa76 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.033585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a2cb2-0804-4821-b19f-8f24190cfa76 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.033585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a2cb2-0804-4821-b19f-8f24190cfa76 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.033585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 97/196] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:44.035954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a2cb2-0804-4821-b19f-8f24190cfa76 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:44.037776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2487dac-820a-42d5-be1b-a69fd56b6171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.038676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2487dac-820a-42d5-be1b-a69fd56b6171 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.038959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2487dac-820a-42d5-be1b-a69fd56b6171 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.039478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2487dac-820a-42d5-be1b-a69fd56b6171 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.040686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2487dac-820a-42d5-be1b-a69fd56b6171 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:44.042205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2487dac-820a-42d5-be1b-a69fd56b6171 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:44.044177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 100/197] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:43 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:44.045233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bd462-88a1-4fbb-b92b-0f7e856284f1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.045519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bd462-88a1-4fbb-b92b-0f7e856284f1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.045818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bd462-88a1-4fbb-b92b-0f7e856284f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.063111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-460bd462-88a1-4fbb-b92b-0f7e856284f1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:44.064051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-460bd462-88a1-4fbb-b92b-0f7e856284f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:44.145906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 101/198] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:44.145906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bd462-88a1-4fbb-b92b-0f7e856284f1 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:44.165209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a2cb2-0804-4821-b19f-8f24190cfa76 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:44.205722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7c7fa-f92f-46a5-819c-34fc5e8c5b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.206633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7c7fa-f92f-46a5-819c-34fc5e8c5b15 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.207596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7c7fa-f92f-46a5-819c-34fc5e8c5b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.207596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7c7fa-f92f-46a5-819c-34fc5e8c5b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.273089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bd462-88a1-4fbb-b92b-0f7e856284f1 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:44.292099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7c7fa-f92f-46a5-819c-34fc5e8c5b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:44.294866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7c7fa-f92f-46a5-819c-34fc5e8c5b15 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:44.314602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.315269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.315715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.315997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.317132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=3bd04aa0eebc4c8390eec35a93467757, user.id=49467f558a2340839aa7c7847f1a5d79) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:44.317444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:44.318926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:44.326932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 102/199] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:44 2022] GET /identity/v3/role_assignments?user.id=49467f558a2340839aa7c7847f1a5d79&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=663341777b0b44128ed78fe8d40f6462 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:44.328028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:44.455692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f7eff-83e5-40da-887c-61d28dee0cd1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:44.487242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 98/200] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:44.491782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb8e53-59d1-4307-839a-23d83f43bfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.492672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8e53-59d1-4307-839a-23d83f43bfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.492958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8e53-59d1-4307-839a-23d83f43bfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.493082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7c7fa-f92f-46a5-819c-34fc5e8c5b15 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:44.493240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb8e53-59d1-4307-839a-23d83f43bfe0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.504642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb8e53-59d1-4307-839a-23d83f43bfe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-67bb8e53-59d1-4307-839a-23d83f43bfe0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:44.513153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:44.514185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:44.515028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 103/201] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:44.521762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb8e53-59d1-4307-839a-23d83f43bfe0 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:44.646998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb8e53-59d1-4307-839a-23d83f43bfe0 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:44.677812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7c7fa-f92f-46a5-819c-34fc5e8c5b15 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:44.689259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.689975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.690450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.690558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.691285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:44.691679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:44.693005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:44.710222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 104/202] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:44.721816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:44.721972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93fdf6-7310-456e-a316-3edbb032c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.722857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93fdf6-7310-456e-a316-3edbb032c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.723139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93fdf6-7310-456e-a316-3edbb032c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.730161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93fdf6-7310-456e-a316-3edbb032c7c6 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.732001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93fdf6-7310-456e-a316-3edbb032c7c6 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:44.733541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93fdf6-7310-456e-a316-3edbb032c7c6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:44.735584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 99/203] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:44 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:44.740897 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93fdf6-7310-456e-a316-3edbb032c7c6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:44.862805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c63f-0f92-4fec-ada0-5bb0f90ee93d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:44.917980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93fdf6-7310-456e-a316-3edbb032c7c6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:44.923940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8f7a6-4743-46c3-9eb8-85bb6ae60f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.924881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f7a6-4743-46c3-9eb8-85bb6ae60f6c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.925197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f7a6-4743-46c3-9eb8-85bb6ae60f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.925446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f7a6-4743-46c3-9eb8-85bb6ae60f6c None admin] PATH_INFO: `/v3/users/8061e5277d3743a3bb68b194cf969b27` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.931220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8f7a6-4743-46c3-9eb8-85bb6ae60f6c None admin] RBAC: Authorizing `identity:get_user(user_id=8061e5277d3743a3bb68b194cf969b27)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:44.932716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8f7a6-4743-46c3-9eb8-85bb6ae60f6c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:44.935251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 105/204] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:44 2022] GET /identity/v3/users/8061e5277d3743a3bb68b194cf969b27 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:44.940548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8f7a6-4743-46c3-9eb8-85bb6ae60f6c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:44.988288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:44.988948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:44.989228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:44.989502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:44.990367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] RBAC: Adding query filter params (role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=3bd04aa0eebc4c8390eec35a93467757, user.id=49467f558a2340839aa7c7847f1a5d79) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:44.990673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:44.992110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:45.005919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 100/205] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:44 2022] GET /identity/v3/role_assignments?user.id=49467f558a2340839aa7c7847f1a5d79&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=9065c36fc305418baa5b977858b445dd => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:45.071458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8f7a6-4743-46c3-9eb8-85bb6ae60f6c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:45.107841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c3303-5ea1-496e-9c32-66607b60fff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:45.108894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3303-5ea1-496e-9c32-66607b60fff4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:45.109388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3303-5ea1-496e-9c32-66607b60fff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:45.109895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c3303-5ea1-496e-9c32-66607b60fff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:45.189541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c3303-5ea1-496e-9c32-66607b60fff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:45.191417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c3303-5ea1-496e-9c32-66607b60fff4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:45.310902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 106/206] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:45.316299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:45.470895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f30404-ed99-49a6-9582-7d4f1c4a0f3c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:45.518484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b54a16-1f14-42b5-b9ae-0edbda4a7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:45.519379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b54a16-1f14-42b5-b9ae-0edbda4a7e65 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:45.519668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b54a16-1f14-42b5-b9ae-0edbda4a7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:45.519948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b54a16-1f14-42b5-b9ae-0edbda4a7e65 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:45.521197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b54a16-1f14-42b5-b9ae-0edbda4a7e65 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:45.522678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b54a16-1f14-42b5-b9ae-0edbda4a7e65 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:45.524749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 101/207] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:45 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:45.531998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c3303-5ea1-496e-9c32-66607b60fff4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:45.672516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c3303-5ea1-496e-9c32-66607b60fff4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:45.719006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6902aec-11e8-4307-8077-5c0f45a8d053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:45.720028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902aec-11e8-4307-8077-5c0f45a8d053 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:45.720518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902aec-11e8-4307-8077-5c0f45a8d053 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:45.721020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902aec-11e8-4307-8077-5c0f45a8d053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:45.722082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902aec-11e8-4307-8077-5c0f45a8d053 None admin] RBAC: Adding query filter params (role.id=1163600e955a4a41905c8cc295ce219b, scope.project.id=1eafe2a57502441faa06c7ab6de43b01, user.id=8061e5277d3743a3bb68b194cf969b27) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:45.722621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902aec-11e8-4307-8077-5c0f45a8d053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:45.724266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902aec-11e8-4307-8077-5c0f45a8d053 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:45.734776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 107/208] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:45 2022] GET /identity/v3/role_assignments?user.id=8061e5277d3743a3bb68b194cf969b27&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=1163600e955a4a41905c8cc295ce219b => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:46.418722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfd77-1c28-4f5a-921d-0baa45b92358 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:46.419237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfd77-1c28-4f5a-921d-0baa45b92358 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:46.419342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cfd77-1c28-4f5a-921d-0baa45b92358 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:46.421032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 102/209] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:46.426405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56909b0c-8f9e-4d75-992d-f395027fa9ba None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:46.426862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56909b0c-8f9e-4d75-992d-f395027fa9ba None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:46.427075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56909b0c-8f9e-4d75-992d-f395027fa9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:46.444719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-56909b0c-8f9e-4d75-992d-f395027fa9ba None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:46.445950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-56909b0c-8f9e-4d75-992d-f395027fa9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:46.519573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 108/210] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:46.542576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cfd77-1c28-4f5a-921d-0baa45b92358 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:46.590547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904665f-1ce2-4080-b6e1-569169ed9e58 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:46.591195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904665f-1ce2-4080-b6e1-569169ed9e58 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:46.591683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904665f-1ce2-4080-b6e1-569169ed9e58 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:46.593284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 109/211] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:46.598577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db340ca9-3481-4d7f-85de-8c71e0d1531b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:46.599109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db340ca9-3481-4d7f-85de-8c71e0d1531b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:46.599619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db340ca9-3481-4d7f-85de-8c71e0d1531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:46.617313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-db340ca9-3481-4d7f-85de-8c71e0d1531b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:46.618876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-db340ca9-3481-4d7f-85de-8c71e0d1531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:46.654376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cfd77-1c28-4f5a-921d-0baa45b92358 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:46.686288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 110/212] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:46.696372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632f200-79f2-4a0b-b169-d8b818d7ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:46.697316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f200-79f2-4a0b-b169-d8b818d7ce0c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:46.697701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f200-79f2-4a0b-b169-d8b818d7ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:46.698107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f200-79f2-4a0b-b169-d8b818d7ce0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:46.709136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632f200-79f2-4a0b-b169-d8b818d7ce0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:46.713680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db340ca9-3481-4d7f-85de-8c71e0d1531b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4632f200-79f2-4a0b-b169-d8b818d7ce0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:46.718391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:46.719738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:46.721332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 103/213] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:46.725491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632f200-79f2-4a0b-b169-d8b818d7ce0c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:46.833301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632f200-79f2-4a0b-b169-d8b818d7ce0c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:46.840564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db340ca9-3481-4d7f-85de-8c71e0d1531b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:46.876642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:46.877834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:46.878441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:46.878994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:46.880188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:46.880771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:46.881285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe25e3-0bf8-4b2a-bf3b-48b73635c9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:46.882214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe25e3-0bf8-4b2a-bf3b-48b73635c9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:46.882569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe25e3-0bf8-4b2a-bf3b-48b73635c9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:46.882918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe25e3-0bf8-4b2a-bf3b-48b73635c9d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:46.883248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:46.884228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe25e3-0bf8-4b2a-bf3b-48b73635c9d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:46.886323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe25e3-0bf8-4b2a-bf3b-48b73635c9d8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:46.888050 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 111/214] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:46.893428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe25e3-0bf8-4b2a-bf3b-48b73635c9d8 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:46.893987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 104/215] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:46.905371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:47.006019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe25e3-0bf8-4b2a-bf3b-48b73635c9d8 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.019489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e1848-93e1-4d2e-962d-20d9f57286b1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.044529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa6144-122e-4ba1-a205-ad2fa01392af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:47.045593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa6144-122e-4ba1-a205-ad2fa01392af None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.046114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa6144-122e-4ba1-a205-ad2fa01392af None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.046622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa6144-122e-4ba1-a205-ad2fa01392af None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.047550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa6144-122e-4ba1-a205-ad2fa01392af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:47.049228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa6144-122e-4ba1-a205-ad2fa01392af None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:47.061732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d2d72-1525-4fe3-bab0-fbfd30ac6e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:47.063063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2d72-1525-4fe3-bab0-fbfd30ac6e78 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.063486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2d72-1525-4fe3-bab0-fbfd30ac6e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.063897 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2d72-1525-4fe3-bab0-fbfd30ac6e78 None admin] PATH_INFO: `/v3/users/a7f466f4a7a0489b8725ea5796caa578` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.083161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d2d72-1525-4fe3-bab0-fbfd30ac6e78 None admin] RBAC: Authorizing `identity:get_user(user_id=a7f466f4a7a0489b8725ea5796caa578)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:47.084751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d2d72-1525-4fe3-bab0-fbfd30ac6e78 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:47.087002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 105/216] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:46 2022] GET /identity/v3/users/a7f466f4a7a0489b8725ea5796caa578 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:47.095156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d2d72-1525-4fe3-bab0-fbfd30ac6e78 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:47.143276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 112/217] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:03:46 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:03:47.204780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d2d72-1525-4fe3-bab0-fbfd30ac6e78 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.217672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ad0fc-1f17-4de0-aeb4-33a76be980af None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.217886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ad0fc-1f17-4de0-aeb4-33a76be980af None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.218170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ad0fc-1f17-4de0-aeb4-33a76be980af None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.219599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 113/218] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:47.225333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9f68c-d0aa-45fb-a387-1bf31a4aa5e8 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.225998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9f68c-d0aa-45fb-a387-1bf31a4aa5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.226482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9f68c-d0aa-45fb-a387-1bf31a4aa5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.245911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffcbcb-52d0-48d1-b963-e95186e7b772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:47.246895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffcbcb-52d0-48d1-b963-e95186e7b772 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.247186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffcbcb-52d0-48d1-b963-e95186e7b772 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.247469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffcbcb-52d0-48d1-b963-e95186e7b772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.247571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-21b9f68c-d0aa-45fb-a387-1bf31a4aa5e8 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:47.249075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-21b9f68c-d0aa-45fb-a387-1bf31a4aa5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:47.323171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffcbcb-52d0-48d1-b963-e95186e7b772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:47.325130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffcbcb-52d0-48d1-b963-e95186e7b772 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:47.351260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 114/219] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:47.375023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9f68c-d0aa-45fb-a387-1bf31a4aa5e8 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:47.443030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 106/220] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:03:47.450355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffcbcb-52d0-48d1-b963-e95186e7b772 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:47.502331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9f68c-d0aa-45fb-a387-1bf31a4aa5e8 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.538245 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6c905-c8ce-4259-a65b-8f056d075025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:47.539058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c905-c8ce-4259-a65b-8f056d075025 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.539341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c905-c8ce-4259-a65b-8f056d075025 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.539616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c905-c8ce-4259-a65b-8f056d075025 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.552682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6c905-c8ce-4259-a65b-8f056d075025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:47.554875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffcbcb-52d0-48d1-b963-e95186e7b772 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-26f6c905-c8ce-4259-a65b-8f056d075025 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:47.561745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:47.562691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:47.563485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 115/221] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:47.566198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17676405-797b-4323-b950-b7986e9a1f22 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.566198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17676405-797b-4323-b950-b7986e9a1f22 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.566198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17676405-797b-4323-b950-b7986e9a1f22 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.566198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 116/222] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:47.571491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49317285-2238-43be-8b1d-3734f143d3f8 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.571782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49317285-2238-43be-8b1d-3734f143d3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.572055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49317285-2238-43be-8b1d-3734f143d3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.589597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-49317285-2238-43be-8b1d-3734f143d3f8 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:47.590736 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-49317285-2238-43be-8b1d-3734f143d3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:47.592361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f10a68-21b2-4c2d-a434-413016b6578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:47.594016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10a68-21b2-4c2d-a434-413016b6578a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.594016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10a68-21b2-4c2d-a434-413016b6578a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.594016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f10a68-21b2-4c2d-a434-413016b6578a None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.596980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f10a68-21b2-4c2d-a434-413016b6578a None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:47.596980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f10a68-21b2-4c2d-a434-413016b6578a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:47.601827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 107/223] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:47 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:47.601827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f10a68-21b2-4c2d-a434-413016b6578a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:47.670606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 117/224] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:47.672198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49317285-2238-43be-8b1d-3734f143d3f8 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:47.719850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f10a68-21b2-4c2d-a434-413016b6578a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.763485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:47.764458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.764942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.765405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.766352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:47.766836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:47.768849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:47.779641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 108/225] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:47.780807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:47.794744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49317285-2238-43be-8b1d-3734f143d3f8 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.835722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:47.836681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:47.837143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:47.837620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:47.838646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] RBAC: Adding query filter params (role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=3bd04aa0eebc4c8390eec35a93467757, user.id=a7f466f4a7a0489b8725ea5796caa578) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:47.839136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:47.840854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:47.846652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 118/226] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:47 2022] GET /identity/v3/role_assignments?user.id=a7f466f4a7a0489b8725ea5796caa578&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=3552540dc0054d2e87fad160a358c58b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:47.847666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:47.931349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfac41f-752b-41c3-8ffd-66ff16213bb1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.974041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b12b14-0ed0-4d14-a45c-7c74af97c323 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:47.999169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da911824-6614-4aac-a4e0-8ae5221af442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:48.000010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da911824-6614-4aac-a4e0-8ae5221af442 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:48.000294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da911824-6614-4aac-a4e0-8ae5221af442 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:48.000573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da911824-6614-4aac-a4e0-8ae5221af442 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:48.013978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781b730-0d89-4cbd-b144-db0dcbc39b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:48.015217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781b730-0d89-4cbd-b144-db0dcbc39b76 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:48.015732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781b730-0d89-4cbd-b144-db0dcbc39b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:48.016202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781b730-0d89-4cbd-b144-db0dcbc39b76 None admin] PATH_INFO: `/v3/users/49467f558a2340839aa7c7847f1a5d79` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:48.017672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781b730-0d89-4cbd-b144-db0dcbc39b76 None admin] RBAC: Authorizing `identity:get_user(user_id=49467f558a2340839aa7c7847f1a5d79)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:48.019522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781b730-0d89-4cbd-b144-db0dcbc39b76 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:48.021468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 119/227] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:47 2022] GET /identity/v3/users/49467f558a2340839aa7c7847f1a5d79 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:48.025817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da911824-6614-4aac-a4e0-8ae5221af442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:48.031887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781b730-0d89-4cbd-b144-db0dcbc39b76 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-da911824-6614-4aac-a4e0-8ae5221af442 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:48.037487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:48.038456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:48.039251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 109/228] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:03:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:48.048322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da911824-6614-4aac-a4e0-8ae5221af442 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:48.201748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da911824-6614-4aac-a4e0-8ae5221af442 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:48.206716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781b730-0d89-4cbd-b144-db0dcbc39b76 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:48.245175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:48.245911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:48.246201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:48.246482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:48.247235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:48.247545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:48.251019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:48.267410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 110/229] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:03:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:48.274942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:48.282091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5493530-0d8c-460f-8d0f-66bf6b83812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:48.283033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5493530-0d8c-460f-8d0f-66bf6b83812d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:48.283324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5493530-0d8c-460f-8d0f-66bf6b83812d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:48.283600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5493530-0d8c-460f-8d0f-66bf6b83812d None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:48.290563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5493530-0d8c-460f-8d0f-66bf6b83812d None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:48.292082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5493530-0d8c-460f-8d0f-66bf6b83812d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:48.293868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 120/230] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:48 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:48.302560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5493530-0d8c-460f-8d0f-66bf6b83812d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:48.413834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3fbe0-264c-43e7-9886-6c9d0b51a19c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:48.457456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdec21-a6a6-4e08-8f77-c369ec0bb3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:48.458443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdec21-a6a6-4e08-8f77-c369ec0bb3ca None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:48.458734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdec21-a6a6-4e08-8f77-c369ec0bb3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:48.459012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdec21-a6a6-4e08-8f77-c369ec0bb3ca None admin] PATH_INFO: `/v3/users/49467f558a2340839aa7c7847f1a5d79` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:48.464139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdec21-a6a6-4e08-8f77-c369ec0bb3ca None admin] RBAC: Authorizing `identity:get_user(user_id=49467f558a2340839aa7c7847f1a5d79)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:48.465660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdec21-a6a6-4e08-8f77-c369ec0bb3ca None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:48.467425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 111/231] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:48 2022] GET /identity/v3/users/49467f558a2340839aa7c7847f1a5d79 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:48.473077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdec21-a6a6-4e08-8f77-c369ec0bb3ca None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:48.481902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5493530-0d8c-460f-8d0f-66bf6b83812d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:48.524358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfa264-1653-4e83-97e1-674fc7433fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:48.525830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfa264-1653-4e83-97e1-674fc7433fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:48.525830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfa264-1653-4e83-97e1-674fc7433fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:48.526142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bfa264-1653-4e83-97e1-674fc7433fe4 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757/users/49467f558a2340839aa7c7847f1a5d79/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:48.538728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfa264-1653-4e83-97e1-674fc7433fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bd04aa0eebc4c8390eec35a93467757, user_id=49467f558a2340839aa7c7847f1a5d79, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:48.540217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bfa264-1653-4e83-97e1-674fc7433fe4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:48.601870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdec21-a6a6-4e08-8f77-c369ec0bb3ca None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:48.624964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 121/232] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:48 2022] PUT /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757/users/49467f558a2340839aa7c7847f1a5d79/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:48.649449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5e23a-9436-467e-9fdd-85cf5b00bec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:48.650458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5e23a-9436-467e-9fdd-85cf5b00bec5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:48.650747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5e23a-9436-467e-9fdd-85cf5b00bec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:48.651027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5e23a-9436-467e-9fdd-85cf5b00bec5 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:48.652964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5e23a-9436-467e-9fdd-85cf5b00bec5 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:48.654534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5e23a-9436-467e-9fdd-85cf5b00bec5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:48.656273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 112/233] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:48 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:48.662748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfa264-1653-4e83-97e1-674fc7433fe4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:48.802762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bfa264-1653-4e83-97e1-674fc7433fe4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:48.855781 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b7173-f5a2-42a1-9d87-43ee35a94730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:48.856700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7173-f5a2-42a1-9d87-43ee35a94730 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:48.856980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7173-f5a2-42a1-9d87-43ee35a94730 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:48.857255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7173-f5a2-42a1-9d87-43ee35a94730 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757/users/49467f558a2340839aa7c7847f1a5d79/roles/9065c36fc305418baa5b977858b445dd` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:48.863750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b7173-f5a2-42a1-9d87-43ee35a94730 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bd04aa0eebc4c8390eec35a93467757, user_id=49467f558a2340839aa7c7847f1a5d79, role_id=9065c36fc305418baa5b977858b445dd)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:48.865231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b7173-f5a2-42a1-9d87-43ee35a94730 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:49.002184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 122/234] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:48 2022] PUT /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757/users/49467f558a2340839aa7c7847f1a5d79/roles/9065c36fc305418baa5b977858b445dd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:49.831081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb8182-d455-4431-b915-a4a4c1d8228e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:49.832418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb8182-d455-4431-b915-a4a4c1d8228e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:49.833145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb8182-d455-4431-b915-a4a4c1d8228e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:49.834857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 113/235] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:49.841821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57483b-bbdc-43e3-865f-315965677e81 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:49.842992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57483b-bbdc-43e3-865f-315965677e81 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:49.843668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57483b-bbdc-43e3-865f-315965677e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:49.864572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ba57483b-bbdc-43e3-865f-315965677e81 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:49.866131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ba57483b-bbdc-43e3-865f-315965677e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:49.934515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 123/236] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:49.971573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb8182-d455-4431-b915-a4a4c1d8228e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:50.091453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347a028-f23f-47fc-99d0-49a3d9f3ab42 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.092202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347a028-f23f-47fc-99d0-49a3d9f3ab42 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.092704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347a028-f23f-47fc-99d0-49a3d9f3ab42 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.094322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 124/237] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:50.107477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb8182-d455-4431-b915-a4a4c1d8228e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:50.112672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691cf1c-3df9-47b1-9c73-d7a95135e90e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.113246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691cf1c-3df9-47b1-9c73-d7a95135e90e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.113723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691cf1c-3df9-47b1-9c73-d7a95135e90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.145828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1691cf1c-3df9-47b1-9c73-d7a95135e90e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:50.145828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1691cf1c-3df9-47b1-9c73-d7a95135e90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:50.190671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad0475-7b9a-4ae8-b2e0-b39078737754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:50.191482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad0475-7b9a-4ae8-b2e0-b39078737754 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.191766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad0475-7b9a-4ae8-b2e0-b39078737754 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.192043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad0475-7b9a-4ae8-b2e0-b39078737754 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.225593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad0475-7b9a-4ae8-b2e0-b39078737754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-08ad0475-7b9a-4ae8-b2e0-b39078737754 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:50.244765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:50.246276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:50.247064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:50.250085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 114/238] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:50.257070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad0475-7b9a-4ae8-b2e0-b39078737754 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:50.291373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 125/239] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:50.315931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691cf1c-3df9-47b1-9c73-d7a95135e90e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:50.450755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad0475-7b9a-4ae8-b2e0-b39078737754 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:50.494478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691cf1c-3df9-47b1-9c73-d7a95135e90e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:50.498139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:50.498139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.498347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.498601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.499303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:50.499607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:50.501001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:50.530856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 115/240] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:50.546290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:50.558758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921d2cf-1bd6-4e61-a11e-0e4bc1bb984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:50.559573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921d2cf-1bd6-4e61-a11e-0e4bc1bb984e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.559859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921d2cf-1bd6-4e61-a11e-0e4bc1bb984e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.560135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921d2cf-1bd6-4e61-a11e-0e4bc1bb984e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.561331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0921d2cf-1bd6-4e61-a11e-0e4bc1bb984e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:50.562814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0921d2cf-1bd6-4e61-a11e-0e4bc1bb984e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:50.569218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 126/241] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:50.570041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921d2cf-1bd6-4e61-a11e-0e4bc1bb984e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:50.692078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda145e-5dfb-49cf-a57b-e632d8723aec None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:50.717086 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921d2cf-1bd6-4e61-a11e-0e4bc1bb984e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:50.734982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b4a0d-2c3e-4f70-84c9-301caa07c2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:50.735921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b4a0d-2c3e-4f70-84c9-301caa07c2ef None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.736210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b4a0d-2c3e-4f70-84c9-301caa07c2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.736488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b4a0d-2c3e-4f70-84c9-301caa07c2ef None admin] PATH_INFO: `/v3/users/bae662443374453981a7ccbb95edc250` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.752090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3aa56-e532-4fb2-8722-399bc948b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:50.752880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3aa56-e532-4fb2-8722-399bc948b811 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.753227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3aa56-e532-4fb2-8722-399bc948b811 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.753680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3aa56-e532-4fb2-8722-399bc948b811 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.754222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b4a0d-2c3e-4f70-84c9-301caa07c2ef None admin] RBAC: Authorizing `identity:get_user(user_id=bae662443374453981a7ccbb95edc250)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:50.754540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3aa56-e532-4fb2-8722-399bc948b811 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:50.755697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b4a0d-2c3e-4f70-84c9-301caa07c2ef None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:50.756091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3aa56-e532-4fb2-8722-399bc948b811 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:50.757593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 116/242] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:50 2022] GET /identity/v3/users/bae662443374453981a7ccbb95edc250 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:50.759002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9693c-2681-4d6d-8292-319e433cf180 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.759297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9693c-2681-4d6d-8292-319e433cf180 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.759580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9693c-2681-4d6d-8292-319e433cf180 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.760801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 117/243] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:50.766303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9693c-2681-4d6d-8292-319e433cf180 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:50.844168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 127/244] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:03:50 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:50.846107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91263f98-ba5c-4a06-893b-e5029b6e41fc None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.846408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91263f98-ba5c-4a06-893b-e5029b6e41fc None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.846745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91263f98-ba5c-4a06-893b-e5029b6e41fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:50.883552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-91263f98-ba5c-4a06-893b-e5029b6e41fc None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:50.887955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-91263f98-ba5c-4a06-893b-e5029b6e41fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:50.908568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9693c-2681-4d6d-8292-319e433cf180 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:50.983217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec32393-b60d-42f6-a7cd-410d6ca45f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:50.983889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32393-b60d-42f6-a7cd-410d6ca45f95 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:50.984172 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32393-b60d-42f6-a7cd-410d6ca45f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:50.984449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32393-b60d-42f6-a7cd-410d6ca45f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.037971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 128/245] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:51.062871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91263f98-ba5c-4a06-893b-e5029b6e41fc None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.088636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec32393-b60d-42f6-a7cd-410d6ca45f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:51.090830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec32393-b60d-42f6-a7cd-410d6ca45f95 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:51.170223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91263f98-ba5c-4a06-893b-e5029b6e41fc None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:51.194181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 118/246] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:51.201887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec32393-b60d-42f6-a7cd-410d6ca45f95 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.222678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2b06d-1a1f-4a61-a09a-5db6d4dfe224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:51.223839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2b06d-1a1f-4a61-a09a-5db6d4dfe224 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.224318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2b06d-1a1f-4a61-a09a-5db6d4dfe224 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.224813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2b06d-1a1f-4a61-a09a-5db6d4dfe224 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.235934 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2b06d-1a1f-4a61-a09a-5db6d4dfe224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-8cc2b06d-1a1f-4a61-a09a-5db6d4dfe224 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:51.246372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:51.247260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:51.248042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:51.250800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 129/247] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:51.257213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2b06d-1a1f-4a61-a09a-5db6d4dfe224 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.312309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec32393-b60d-42f6-a7cd-410d6ca45f95 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:51.354509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091323a-ede4-48e4-a945-184e393dbcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:51.355936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091323a-ede4-48e4-a945-184e393dbcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.356270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091323a-ede4-48e4-a945-184e393dbcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.356490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091323a-ede4-48e4-a945-184e393dbcc9 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.358587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091323a-ede4-48e4-a945-184e393dbcc9 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:51.360060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091323a-ede4-48e4-a945-184e393dbcc9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:51.362027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 119/248] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:51 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:51.363856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb53e6-80f4-4921-bcf0-56a3c99931f3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.364151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb53e6-80f4-4921-bcf0-56a3c99931f3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.364433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb53e6-80f4-4921-bcf0-56a3c99931f3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.365681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 120/249] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:51.370491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb53e6-80f4-4921-bcf0-56a3c99931f3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.375689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2b06d-1a1f-4a61-a09a-5db6d4dfe224 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:51.410367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a92c5-a6d6-4c40-9ce4-46d1f7e2442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:51.411087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a92c5-a6d6-4c40-9ce4-46d1f7e2442f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.411367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a92c5-a6d6-4c40-9ce4-46d1f7e2442f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.411689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a92c5-a6d6-4c40-9ce4-46d1f7e2442f None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.412439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a92c5-a6d6-4c40-9ce4-46d1f7e2442f None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:51.412741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a92c5-a6d6-4c40-9ce4-46d1f7e2442f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:51.414244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a92c5-a6d6-4c40-9ce4-46d1f7e2442f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:51.426354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 130/250] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:51.426517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac3cd7-3d5c-494d-b153-b48959df65c3 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.426786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac3cd7-3d5c-494d-b153-b48959df65c3 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.427079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac3cd7-3d5c-494d-b153-b48959df65c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.452381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-57ac3cd7-3d5c-494d-b153-b48959df65c3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:51.454120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-57ac3cd7-3d5c-494d-b153-b48959df65c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:51.505378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb53e6-80f4-4921-bcf0-56a3c99931f3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:51.544456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 131/251] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:51.547646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac3cd7-3d5c-494d-b153-b48959df65c3 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.550682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:51.554395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.554395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.554395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.554395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] RBAC: Adding query filter params (role.id=1163600e955a4a41905c8cc295ce219b, scope.project.id=3bd04aa0eebc4c8390eec35a93467757, user.id=bae662443374453981a7ccbb95edc250) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:51.554395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:51.555193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:51.562803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 121/252] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:51 2022] GET /identity/v3/role_assignments?user.id=bae662443374453981a7ccbb95edc250&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=1163600e955a4a41905c8cc295ce219b => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:51.581546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.679425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac3cd7-3d5c-494d-b153-b48959df65c3 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:51.708248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6485c-8f5b-4433-8910-18f2be059a8d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:51.727636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5012f-85c8-4c99-a6e5-34567a8d5ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:51.728863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5012f-85c8-4c99-a6e5-34567a8d5ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.729328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5012f-85c8-4c99-a6e5-34567a8d5ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.729801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5012f-85c8-4c99-a6e5-34567a8d5ed6 None admin] PATH_INFO: `/v3/users/a7f466f4a7a0489b8725ea5796caa578` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.731273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5012f-85c8-4c99-a6e5-34567a8d5ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=a7f466f4a7a0489b8725ea5796caa578)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:51.732944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5012f-85c8-4c99-a6e5-34567a8d5ed6 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:51.734907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 132/253] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:51 2022] GET /identity/v3/users/a7f466f4a7a0489b8725ea5796caa578 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:51.741866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5012f-85c8-4c99-a6e5-34567a8d5ed6 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.744179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911c4ea-d150-4472-9c5e-ee8816b9c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:51.744979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911c4ea-d150-4472-9c5e-ee8816b9c95f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.745260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911c4ea-d150-4472-9c5e-ee8816b9c95f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.746192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911c4ea-d150-4472-9c5e-ee8816b9c95f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.761889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911c4ea-d150-4472-9c5e-ee8816b9c95f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7911c4ea-d150-4472-9c5e-ee8816b9c95f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:51.765333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:51.766377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:51.767333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 122/254] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:03:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:51.772679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911c4ea-d150-4472-9c5e-ee8816b9c95f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.915138 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911c4ea-d150-4472-9c5e-ee8816b9c95f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:51.915740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5012f-85c8-4c99-a6e5-34567a8d5ed6 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:51.952763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea66b7-eedb-4d70-9b9e-88c7ba1bbde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:51.953641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea66b7-eedb-4d70-9b9e-88c7ba1bbde9 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.953977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea66b7-eedb-4d70-9b9e-88c7ba1bbde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.954227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea66b7-eedb-4d70-9b9e-88c7ba1bbde9 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.955358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b36564-5fc2-47dc-a474-bc614e871407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:51.956022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b36564-5fc2-47dc-a474-bc614e871407 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.956300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b36564-5fc2-47dc-a474-bc614e871407 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.956575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b36564-5fc2-47dc-a474-bc614e871407 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.957328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b36564-5fc2-47dc-a474-bc614e871407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:51.957633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b36564-5fc2-47dc-a474-bc614e871407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:51.958572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea66b7-eedb-4d70-9b9e-88c7ba1bbde9 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:51.959064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b36564-5fc2-47dc-a474-bc614e871407 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:51.960712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea66b7-eedb-4d70-9b9e-88c7ba1bbde9 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:51.963441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 133/255] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:51 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:51.965457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ff12-6532-418e-abf0-bbe43051d5cf None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.966904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ff12-6532-418e-abf0-bbe43051d5cf None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.967398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ff12-6532-418e-abf0-bbe43051d5cf None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:51.969215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 134/256] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:51.969676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 123/257] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:51.970279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7ff12-6532-418e-abf0-bbe43051d5cf None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:51.975013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3a0a8-b678-4aca-8c32-2e2719896480 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:51.975509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3a0a8-b678-4aca-8c32-2e2719896480 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:51.975982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3a0a8-b678-4aca-8c32-2e2719896480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:52.004723 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-35b3a0a8-b678-4aca-8c32-2e2719896480 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:52.008102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-35b3a0a8-b678-4aca-8c32-2e2719896480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:52.137125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 124/258] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:52.138316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3a0a8-b678-4aca-8c32-2e2719896480 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:52.160177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7ff12-6532-418e-abf0-bbe43051d5cf None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:52.220744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116f803-4443-4f85-a9bb-a5ed5c57d9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:52.222055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116f803-4443-4f85-a9bb-a5ed5c57d9f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:52.222541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116f803-4443-4f85-a9bb-a5ed5c57d9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:52.223013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116f803-4443-4f85-a9bb-a5ed5c57d9f1 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757/users/a7f466f4a7a0489b8725ea5796caa578/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:52.225389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116f803-4443-4f85-a9bb-a5ed5c57d9f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bd04aa0eebc4c8390eec35a93467757, user_id=a7f466f4a7a0489b8725ea5796caa578, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:52.227186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116f803-4443-4f85-a9bb-a5ed5c57d9f1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:52.279733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3a0a8-b678-4aca-8c32-2e2719896480 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:52.298952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 135/259] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:51 2022] PUT /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757/users/a7f466f4a7a0489b8725ea5796caa578/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:52.305818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116f803-4443-4f85-a9bb-a5ed5c57d9f1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:52.331299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf102df-7b96-4616-a0fb-c3cc1d7d0580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:52.332919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf102df-7b96-4616-a0fb-c3cc1d7d0580 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:52.332919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf102df-7b96-4616-a0fb-c3cc1d7d0580 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:52.332919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf102df-7b96-4616-a0fb-c3cc1d7d0580 None admin] PATH_INFO: `/v3/users/49467f558a2340839aa7c7847f1a5d79` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:52.334114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf102df-7b96-4616-a0fb-c3cc1d7d0580 None admin] RBAC: Authorizing `identity:get_user(user_id=49467f558a2340839aa7c7847f1a5d79)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:52.335602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf102df-7b96-4616-a0fb-c3cc1d7d0580 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:52.337594 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 125/260] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:52 2022] GET /identity/v3/users/49467f558a2340839aa7c7847f1a5d79 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:52.345857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf102df-7b96-4616-a0fb-c3cc1d7d0580 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:52.517645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116f803-4443-4f85-a9bb-a5ed5c57d9f1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:52.564004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf102df-7b96-4616-a0fb-c3cc1d7d0580 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:52.582569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504a8e0-737e-41f6-b87d-6f895c5381f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:52.583389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504a8e0-737e-41f6-b87d-6f895c5381f3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:52.583672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504a8e0-737e-41f6-b87d-6f895c5381f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:52.583940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504a8e0-737e-41f6-b87d-6f895c5381f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:52.604514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504a8e0-737e-41f6-b87d-6f895c5381f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-5504a8e0-737e-41f6-b87d-6f895c5381f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:52.613360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:52.614345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:03:52.615140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:52.616237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 136/261] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:03:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:52.627073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504a8e0-737e-41f6-b87d-6f895c5381f3 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:52.633299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c2b95-a0dc-42de-a8d5-fe56f1c652f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:52.633987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2b95-a0dc-42de-a8d5-fe56f1c652f1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:52.634270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2b95-a0dc-42de-a8d5-fe56f1c652f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:52.634546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2b95-a0dc-42de-a8d5-fe56f1c652f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:52.747256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c2b95-a0dc-42de-a8d5-fe56f1c652f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:52.748643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c2b95-a0dc-42de-a8d5-fe56f1c652f1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:52.802713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504a8e0-737e-41f6-b87d-6f895c5381f3 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:52.879540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:52.880218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:52.880503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:52.880779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:52.881534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:52.881862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:52.883276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:52.899816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 137/262] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:03:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:52.905068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 126/263] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:03:52.908631 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:52.910871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c2b95-a0dc-42de-a8d5-fe56f1c652f1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:53.087360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64c00d-d2be-4206-8ff2-5ba93af19849 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:53.104305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c2b95-a0dc-42de-a8d5-fe56f1c652f1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:53.139848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d9ec7-8d10-45d4-af97-cc57825a31f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:53.141082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d9ec7-8d10-45d4-af97-cc57825a31f3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:53.141575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d9ec7-8d10-45d4-af97-cc57825a31f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:53.142070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d9ec7-8d10-45d4-af97-cc57825a31f3 None admin] PATH_INFO: `/v3/users/49467f558a2340839aa7c7847f1a5d79` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:53.145185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ef3aa-5ce7-45a0-86ba-713f6bd015ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:53.146316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef3aa-5ce7-45a0-86ba-713f6bd015ad None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:53.146490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d9ec7-8d10-45d4-af97-cc57825a31f3 None admin] RBAC: Authorizing `identity:get_user(user_id=49467f558a2340839aa7c7847f1a5d79)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:53.146883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef3aa-5ce7-45a0-86ba-713f6bd015ad None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:53.147361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ef3aa-5ce7-45a0-86ba-713f6bd015ad None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:53.148127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d9ec7-8d10-45d4-af97-cc57825a31f3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:53.148956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ef3aa-5ce7-45a0-86ba-713f6bd015ad None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:53.150126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 138/264] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:52 2022] GET /identity/v3/users/49467f558a2340839aa7c7847f1a5d79 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:53.150875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ef3aa-5ce7-45a0-86ba-713f6bd015ad None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:53.160102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 127/265] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:52 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:53.165362 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d9ec7-8d10-45d4-af97-cc57825a31f3 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:53.165714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ef3aa-5ce7-45a0-86ba-713f6bd015ad None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:53.311991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ef3aa-5ce7-45a0-86ba-713f6bd015ad None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:53.335404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d9ec7-8d10-45d4-af97-cc57825a31f3 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:53.380344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15ca52-8990-40ca-a228-3a15a9e12f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:53.380969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ca52-8990-40ca-a228-3a15a9e12f7b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:53.381255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ca52-8990-40ca-a228-3a15a9e12f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:53.381583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ca52-8990-40ca-a228-3a15a9e12f7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:53.408940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:53.409611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:53.409922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:53.410219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:53.411038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=3bd04aa0eebc4c8390eec35a93467757, user.id=49467f558a2340839aa7c7847f1a5d79) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:53.411352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:53.412794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:53.432174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 139/266] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:53 2022] GET /identity/v3/role_assignments?user.id=49467f558a2340839aa7c7847f1a5d79&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=663341777b0b44128ed78fe8d40f6462 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:53.507980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15ca52-8990-40ca-a228-3a15a9e12f7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:53.509469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15ca52-8990-40ca-a228-3a15a9e12f7b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:53.645604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 128/267] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:53.651365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:53.773634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaba7e-7537-4568-8710-4f17ccfa6a35 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:53.808620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d296697-9ec0-4663-9b5b-7528d311158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:53.809501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d296697-9ec0-4663-9b5b-7528d311158a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:53.809809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d296697-9ec0-4663-9b5b-7528d311158a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:53.810154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d296697-9ec0-4663-9b5b-7528d311158a None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:53.811481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d296697-9ec0-4663-9b5b-7528d311158a None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:53.813175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d296697-9ec0-4663-9b5b-7528d311158a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:53.815146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 140/268] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:53 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:53.820999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15ca52-8990-40ca-a228-3a15a9e12f7b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:53.928020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15ca52-8990-40ca-a228-3a15a9e12f7b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:53.979560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e479ad4-ba63-4edb-ac02-6f95443e8e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:53.980257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e479ad4-ba63-4edb-ac02-6f95443e8e5b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:53.980540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e479ad4-ba63-4edb-ac02-6f95443e8e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:53.980813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e479ad4-ba63-4edb-ac02-6f95443e8e5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:53.981654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e479ad4-ba63-4edb-ac02-6f95443e8e5b None admin] RBAC: Adding query filter params (role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=3bd04aa0eebc4c8390eec35a93467757, user.id=49467f558a2340839aa7c7847f1a5d79) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:53.981996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e479ad4-ba63-4edb-ac02-6f95443e8e5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:53.983507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e479ad4-ba63-4edb-ac02-6f95443e8e5b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:53.991742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 129/269] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:53 2022] GET /identity/v3/role_assignments?user.id=49467f558a2340839aa7c7847f1a5d79&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=9065c36fc305418baa5b977858b445dd => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:54.253623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f965f-3c7b-4458-ab7b-32c039a6347a None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.254158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f965f-3c7b-4458-ab7b-32c039a6347a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.254231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f965f-3c7b-4458-ab7b-32c039a6347a None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.255992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 141/270] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:54.261595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5f3b4-a2a0-4cd5-8c02-c3c5e7451c25 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.262541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5f3b4-a2a0-4cd5-8c02-c3c5e7451c25 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.263031 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5f3b4-a2a0-4cd5-8c02-c3c5e7451c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.285553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-84d5f3b4-a2a0-4cd5-8c02-c3c5e7451c25 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:54.287152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-84d5f3b4-a2a0-4cd5-8c02-c3c5e7451c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:54.391362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 130/271] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:54.428358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f965f-3c7b-4458-ab7b-32c039a6347a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:54.537778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f965f-3c7b-4458-ab7b-32c039a6347a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:54.562859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a6e3-8304-46d8-a01c-098966732db1 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.563476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a6e3-8304-46d8-a01c-098966732db1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.563959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a6e3-8304-46d8-a01c-098966732db1 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.565553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 131/272] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:54.571145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a77c5c-0507-495a-90e7-2533d4c892d0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.571731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a77c5c-0507-495a-90e7-2533d4c892d0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.572214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a77c5c-0507-495a-90e7-2533d4c892d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.573271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca2658-4c6a-4099-9924-d0645b25185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:54.574187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca2658-4c6a-4099-9924-d0645b25185d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.574457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca2658-4c6a-4099-9924-d0645b25185d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.574788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca2658-4c6a-4099-9924-d0645b25185d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.587356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca2658-4c6a-4099-9924-d0645b25185d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-2dca2658-4c6a-4099-9924-d0645b25185d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:54.597081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:54.598035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:54.598829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 142/273] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:54.599959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f583429-b2df-4106-971e-5efc6cc22cd4 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.599959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f583429-b2df-4106-971e-5efc6cc22cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.600182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f583429-b2df-4106-971e-5efc6cc22cd4 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.601136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-09a77c5c-0507-495a-90e7-2533d4c892d0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:54.601653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 143/274] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:54.602754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-09a77c5c-0507-495a-90e7-2533d4c892d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:54.607046 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede60e6-7bad-4575-9a4a-d98b627be85c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.607332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede60e6-7bad-4575-9a4a-d98b627be85c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.607607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede60e6-7bad-4575-9a4a-d98b627be85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.630236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0ede60e6-7bad-4575-9a4a-d98b627be85c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:54.631494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0ede60e6-7bad-4575-9a4a-d98b627be85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:54.685837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 132/275] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:54.685837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a77c5c-0507-495a-90e7-2533d4c892d0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:54.715430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 144/276] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:54.717146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede60e6-7bad-4575-9a4a-d98b627be85c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:54.785001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a77c5c-0507-495a-90e7-2533d4c892d0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:54.816725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede60e6-7bad-4575-9a4a-d98b627be85c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:54.819511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:54.820188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.820470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.820745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.821697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:54.822038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:54.823433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:54.832218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 133/277] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:54.833183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:54.856737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcec4ea-1b78-40e8-ade3-9fbaa5ebfe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:54.857532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcec4ea-1b78-40e8-ade3-9fbaa5ebfe67 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.857860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcec4ea-1b78-40e8-ade3-9fbaa5ebfe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.858138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcec4ea-1b78-40e8-ade3-9fbaa5ebfe67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.859303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcec4ea-1b78-40e8-ade3-9fbaa5ebfe67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:54.860771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcec4ea-1b78-40e8-ade3-9fbaa5ebfe67 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:54.862588 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 145/278] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:54.866968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcec4ea-1b78-40e8-ade3-9fbaa5ebfe67 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:54.932741 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c73cf3-823d-4655-ae19-393e3e7da89f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:54.962220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcec4ea-1b78-40e8-ade3-9fbaa5ebfe67 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:54.966393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677e70e-4472-40f6-8e8f-be1245d97f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:54.967361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677e70e-4472-40f6-8e8f-be1245d97f9e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:54.967814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677e70e-4472-40f6-8e8f-be1245d97f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:54.968239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677e70e-4472-40f6-8e8f-be1245d97f9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:54.978319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677e70e-4472-40f6-8e8f-be1245d97f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a677e70e-4472-40f6-8e8f-be1245d97f9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:54.986787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:54.987844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:54.988840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:54.990943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 134/279] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:54.991954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677e70e-4472-40f6-8e8f-be1245d97f9e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.000192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8c45d-b02e-49f8-82b0-75745707eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.001057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8c45d-b02e-49f8-82b0-75745707eb5c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.001342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8c45d-b02e-49f8-82b0-75745707eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.001620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8c45d-b02e-49f8-82b0-75745707eb5c None admin] PATH_INFO: `/v3/users/5ec73f7bd93146458ec5bcc804b53ec4` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.015342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8c45d-b02e-49f8-82b0-75745707eb5c None admin] RBAC: Authorizing `identity:get_user(user_id=5ec73f7bd93146458ec5bcc804b53ec4)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:55.016896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8c45d-b02e-49f8-82b0-75745707eb5c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:55.018650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 146/280] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:54 2022] GET /identity/v3/users/5ec73f7bd93146458ec5bcc804b53ec4 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:55.019414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8c45d-b02e-49f8-82b0-75745707eb5c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.095955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677e70e-4472-40f6-8e8f-be1245d97f9e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:55.117594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8c45d-b02e-49f8-82b0-75745707eb5c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:55.132496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30e6f-0b62-432d-84b9-74fe20f11e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.133186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30e6f-0b62-432d-84b9-74fe20f11e98 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.133462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30e6f-0b62-432d-84b9-74fe20f11e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.133730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30e6f-0b62-432d-84b9-74fe20f11e98 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.134484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b30e6f-0b62-432d-84b9-74fe20f11e98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:55.136559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b30e6f-0b62-432d-84b9-74fe20f11e98 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:55.157519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.159494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.159494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.159494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.159772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:55.159924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:55.161307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:55.169584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 147/281] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:55.174214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.224218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 135/282] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:03:54 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:55.224218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30e6f-0b62-432d-84b9-74fe20f11e98 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.280339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f445-2758-4e99-9c82-e8d3b6a7d42a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:55.323038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa7fd9-c089-42ff-a458-22411c9c50b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.323753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa7fd9-c089-42ff-a458-22411c9c50b7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.324580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa7fd9-c089-42ff-a458-22411c9c50b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.325879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa7fd9-c089-42ff-a458-22411c9c50b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.325879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa7fd9-c089-42ff-a458-22411c9c50b7 None admin] RBAC: Adding query filter params (role.id=3552540dc0054d2e87fad160a358c58b, user.id=5ec73f7bd93146458ec5bcc804b53ec4, scope.system=all) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:55.326149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa7fd9-c089-42ff-a458-22411c9c50b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:55.327600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa7fd9-c089-42ff-a458-22411c9c50b7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:55.333326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 148/283] 10.222.0.25 () {56 vars in 1308 bytes} [Fri May 13 13:03:55 2022] GET /identity/v3/role_assignments?user.id=5ec73f7bd93146458ec5bcc804b53ec4&scope.system=all&role.id=3552540dc0054d2e87fad160a358c58b => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:55.334499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e44bd9-41b4-48f2-95f1-9b520add5eff None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.334849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e44bd9-41b4-48f2-95f1-9b520add5eff None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.335282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e44bd9-41b4-48f2-95f1-9b520add5eff None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.336544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 149/284] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:55.346125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac67a4-3cc9-4f74-a722-d97b64a20262 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.346125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac67a4-3cc9-4f74-a722-d97b64a20262 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.346125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac67a4-3cc9-4f74-a722-d97b64a20262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.360775 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-40ac67a4-3cc9-4f74-a722-d97b64a20262 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:55.362125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-40ac67a4-3cc9-4f74-a722-d97b64a20262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:55.367344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30e6f-0b62-432d-84b9-74fe20f11e98 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:55.408963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7ca1-2a5b-4a3a-ba8d-ac2b8a274cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.410298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7ca1-2a5b-4a3a-ba8d-ac2b8a274cbc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.410742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7ca1-2a5b-4a3a-ba8d-ac2b8a274cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.411171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7ca1-2a5b-4a3a-ba8d-ac2b8a274cbc None admin] PATH_INFO: `/v3/users/bae662443374453981a7ccbb95edc250` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.412760 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c7ca1-2a5b-4a3a-ba8d-ac2b8a274cbc None admin] RBAC: Authorizing `identity:get_user(user_id=bae662443374453981a7ccbb95edc250)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:55.414614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c7ca1-2a5b-4a3a-ba8d-ac2b8a274cbc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:55.416811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 136/285] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:55 2022] GET /identity/v3/users/bae662443374453981a7ccbb95edc250 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:55.423108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7ca1-2a5b-4a3a-ba8d-ac2b8a274cbc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.446436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 150/286] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:55.464010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac67a4-3cc9-4f74-a722-d97b64a20262 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.592504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7ca1-2a5b-4a3a-ba8d-ac2b8a274cbc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:55.632661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac67a4-3cc9-4f74-a722-d97b64a20262 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:55.646792 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b7b6c-d651-452d-ae55-2f4d5ccb6f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.648585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b7b6c-d651-452d-ae55-2f4d5ccb6f22 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.648941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b7b6c-d651-452d-ae55-2f4d5ccb6f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.649298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b7b6c-d651-452d-ae55-2f4d5ccb6f22 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.650863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b7b6c-d651-452d-ae55-2f4d5ccb6f22 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:55.653559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b7b6c-d651-452d-ae55-2f4d5ccb6f22 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:55.654869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 137/287] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:55 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:55.660754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b7b6c-d651-452d-ae55-2f4d5ccb6f22 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.680083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac413930-8b92-436f-9704-95e3529805da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.681257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac413930-8b92-436f-9704-95e3529805da None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.681759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac413930-8b92-436f-9704-95e3529805da None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.682263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac413930-8b92-436f-9704-95e3529805da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.696574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac413930-8b92-436f-9704-95e3529805da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-ac413930-8b92-436f-9704-95e3529805da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:55.707008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:55.707999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:55.708845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:55.711360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 151/288] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:55.719823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac413930-8b92-436f-9704-95e3529805da None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.791156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b7b6c-d651-452d-ae55-2f4d5ccb6f22 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:55.847845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a20f6-aca0-4039-9e22-814ee335d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.848761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a20f6-aca0-4039-9e22-814ee335d707 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.849049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a20f6-aca0-4039-9e22-814ee335d707 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.849328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a20f6-aca0-4039-9e22-814ee335d707 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757/users/bae662443374453981a7ccbb95edc250/roles/1163600e955a4a41905c8cc295ce219b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.850239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac413930-8b92-436f-9704-95e3529805da None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:55.851444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a20f6-aca0-4039-9e22-814ee335d707 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bd04aa0eebc4c8390eec35a93467757, user_id=bae662443374453981a7ccbb95edc250, role_id=1163600e955a4a41905c8cc295ce219b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:55.853149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a20f6-aca0-4039-9e22-814ee335d707 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:55.891826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:55.892828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:55.893314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:55.893764 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:55.894701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:55.895248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:55.899744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:55.908121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 152/289] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:55.914158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:55.957466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 138/290] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:03:55 2022] PUT /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757/users/bae662443374453981a7ccbb95edc250/roles/1163600e955a4a41905c8cc295ce219b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:56.051386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5f8b-2ce1-4c09-b1d7-a1458a1ea134 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:56.111558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2ef6a-1841-4ec8-a947-eef9de81569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:56.112467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ef6a-1841-4ec8-a947-eef9de81569e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:56.112753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ef6a-1841-4ec8-a947-eef9de81569e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:56.113033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ef6a-1841-4ec8-a947-eef9de81569e None admin] PATH_INFO: `/v3/users/a7f466f4a7a0489b8725ea5796caa578` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:56.118784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2ef6a-1841-4ec8-a947-eef9de81569e None admin] RBAC: Authorizing `identity:get_user(user_id=a7f466f4a7a0489b8725ea5796caa578)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:56.120381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2ef6a-1841-4ec8-a947-eef9de81569e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:56.122590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 153/291] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:55 2022] GET /identity/v3/users/a7f466f4a7a0489b8725ea5796caa578 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:56.127938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a20f6-aca0-4039-9e22-814ee335d707 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:56.249607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a20f6-aca0-4039-9e22-814ee335d707 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:56.286010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c5e9-989f-4bae-a2df-55e7754cf205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:56.286720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c5e9-989f-4bae-a2df-55e7754cf205 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:56.287006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c5e9-989f-4bae-a2df-55e7754cf205 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:56.287283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c5e9-989f-4bae-a2df-55e7754cf205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:56.356042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74c5e9-989f-4bae-a2df-55e7754cf205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:56.357531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74c5e9-989f-4bae-a2df-55e7754cf205 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:56.455671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 139/292] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:56.461312 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2ef6a-1841-4ec8-a947-eef9de81569e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:56.563850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2ef6a-1841-4ec8-a947-eef9de81569e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:56.598101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3975022-0bbf-41ad-8885-347d01f7807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:56.598981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3975022-0bbf-41ad-8885-347d01f7807e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:56.599323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3975022-0bbf-41ad-8885-347d01f7807e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:56.599598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3975022-0bbf-41ad-8885-347d01f7807e None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:56.600833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3975022-0bbf-41ad-8885-347d01f7807e None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:56.602311 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3975022-0bbf-41ad-8885-347d01f7807e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:56.604015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 154/293] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:56 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:03:56.610456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c5e9-989f-4bae-a2df-55e7754cf205 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:56.724065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c5e9-989f-4bae-a2df-55e7754cf205 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:56.758340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15003db-2ae9-43cd-87c2-5a36efd4d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:56.759001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15003db-2ae9-43cd-87c2-5a36efd4d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:56.759290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15003db-2ae9-43cd-87c2-5a36efd4d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:56.759607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15003db-2ae9-43cd-87c2-5a36efd4d8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:56.760449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15003db-2ae9-43cd-87c2-5a36efd4d8e5 None admin] RBAC: Adding query filter params (role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=3bd04aa0eebc4c8390eec35a93467757, user.id=a7f466f4a7a0489b8725ea5796caa578) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:56.760804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15003db-2ae9-43cd-87c2-5a36efd4d8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:56.762855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15003db-2ae9-43cd-87c2-5a36efd4d8e5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:56.769268 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 140/294] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:03:56 2022] GET /identity/v3/role_assignments?user.id=a7f466f4a7a0489b8725ea5796caa578&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=3552540dc0054d2e87fad160a358c58b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:57.759347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3d62-a7ed-4a56-a3c0-dbbba6d5732e None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:57.759836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3d62-a7ed-4a56-a3c0-dbbba6d5732e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:57.759836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b3d62-a7ed-4a56-a3c0-dbbba6d5732e None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:57.761470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 155/295] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:57.766853 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463992a-33e1-4bc8-8958-b46c56be33a0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:57.767378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463992a-33e1-4bc8-8958-b46c56be33a0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:57.767543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463992a-33e1-4bc8-8958-b46c56be33a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:57.787801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b463992a-33e1-4bc8-8958-b46c56be33a0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:57.788988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b463992a-33e1-4bc8-8958-b46c56be33a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:57.856442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 141/296] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:57.878212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b3d62-a7ed-4a56-a3c0-dbbba6d5732e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:57.979083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b3d62-a7ed-4a56-a3c0-dbbba6d5732e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:58.015364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234a9b1-ec2e-45f7-ba25-a836d262e85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:58.016155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a9b1-ec2e-45f7-ba25-a836d262e85c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.016489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a9b1-ec2e-45f7-ba25-a836d262e85c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.016772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a9b1-ec2e-45f7-ba25-a836d262e85c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.028865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234a9b1-ec2e-45f7-ba25-a836d262e85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-a234a9b1-ec2e-45f7-ba25-a836d262e85c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:58.036841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:58.039048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:58.041008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 156/297] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:58.047195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463992a-33e1-4bc8-8958-b46c56be33a0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:58.152084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463992a-33e1-4bc8-8958-b46c56be33a0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:58.185820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e704443-7eb2-450f-b9b0-8a9547de3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:58.186515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e704443-7eb2-450f-b9b0-8a9547de3fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.186798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e704443-7eb2-450f-b9b0-8a9547de3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.187073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e704443-7eb2-450f-b9b0-8a9547de3fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.187825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e704443-7eb2-450f-b9b0-8a9547de3fe6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:58.188127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e704443-7eb2-450f-b9b0-8a9547de3fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:58.189624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e704443-7eb2-450f-b9b0-8a9547de3fe6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:58.198542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 142/298] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:58.204106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234a9b1-ec2e-45f7-ba25-a836d262e85c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:58.233668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559dd7d-d3d1-4564-bdd9-cb4f7637e290 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.233988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559dd7d-d3d1-4564-bdd9-cb4f7637e290 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.234266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559dd7d-d3d1-4564-bdd9-cb4f7637e290 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.236687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 143/299] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:03:58.242386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff86e-1a8c-4ab5-8a13-e8d753ee5f1c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.242675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff86e-1a8c-4ab5-8a13-e8d753ee5f1c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.242954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff86e-1a8c-4ab5-8a13-e8d753ee5f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.260248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-576ff86e-1a8c-4ab5-8a13-e8d753ee5f1c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:58.261507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-576ff86e-1a8c-4ab5-8a13-e8d753ee5f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:58.311810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234a9b1-ec2e-45f7-ba25-a836d262e85c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:58.330044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 144/300] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:58.331715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aacda-9846-427f-82bd-59bceb47ed76 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.332071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aacda-9846-427f-82bd-59bceb47ed76 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.332425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aacda-9846-427f-82bd-59bceb47ed76 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.338088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 145/301] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:58.339445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657502a-dc30-405d-aa4f-1bc55d831e86 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.339812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657502a-dc30-405d-aa4f-1bc55d831e86 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.340169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657502a-dc30-405d-aa4f-1bc55d831e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.357840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c26bc4-b597-43d9-a0e6-cc0cb68fd940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:58.358715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26bc4-b597-43d9-a0e6-cc0cb68fd940 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.359001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26bc4-b597-43d9-a0e6-cc0cb68fd940 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.359280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c26bc4-b597-43d9-a0e6-cc0cb68fd940 None admin] PATH_INFO: `/v3/users/5ec73f7bd93146458ec5bcc804b53ec4` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.359971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e657502a-dc30-405d-aa4f-1bc55d831e86 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:58.360598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c26bc4-b597-43d9-a0e6-cc0cb68fd940 None admin] RBAC: Authorizing `identity:get_user(user_id=5ec73f7bd93146458ec5bcc804b53ec4)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:58.361876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e657502a-dc30-405d-aa4f-1bc55d831e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:58.362123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c26bc4-b597-43d9-a0e6-cc0cb68fd940 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:58.368003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 157/302] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:58 2022] GET /identity/v3/users/5ec73f7bd93146458ec5bcc804b53ec4 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:58.368003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c26bc4-b597-43d9-a0e6-cc0cb68fd940 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:58.431495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 146/303] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:03:58.432732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657502a-dc30-405d-aa4f-1bc55d831e86 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:58.485471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c26bc4-b597-43d9-a0e6-cc0cb68fd940 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:58.520358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048569d-bbab-42ff-a692-cee236a51aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:58.521168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048569d-bbab-42ff-a692-cee236a51aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.521438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048569d-bbab-42ff-a692-cee236a51aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.521713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048569d-bbab-42ff-a692-cee236a51aa2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.532067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048569d-bbab-42ff-a692-cee236a51aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:58.535408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657502a-dc30-405d-aa4f-1bc55d831e86 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-d048569d-bbab-42ff-a692-cee236a51aa2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:58.539635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:58.540520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:58.541315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:03:58.545648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 158/304] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:03:58.546302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048569d-bbab-42ff-a692-cee236a51aa2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:58.569154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24b7f3-5465-4821-87a3-41088f1205de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:58.569975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24b7f3-5465-4821-87a3-41088f1205de None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.570264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24b7f3-5465-4821-87a3-41088f1205de None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.570536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24b7f3-5465-4821-87a3-41088f1205de None admin] PATH_INFO: `/v3/system/users/5ec73f7bd93146458ec5bcc804b53ec4/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.572029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24b7f3-5465-4821-87a3-41088f1205de None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5ec73f7bd93146458ec5bcc804b53ec4, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:58.573974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24b7f3-5465-4821-87a3-41088f1205de None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:58.642768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048569d-bbab-42ff-a692-cee236a51aa2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:58.669638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 147/305] 10.222.0.25 () {58 vars in 1334 bytes} [Fri May 13 13:03:58 2022] PUT /identity/v3/system/users/5ec73f7bd93146458ec5bcc804b53ec4/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:03:58.670431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24b7f3-5465-4821-87a3-41088f1205de None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:58.678054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84072c63-2bd0-420f-8426-cbdafe33c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:58.678886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84072c63-2bd0-420f-8426-cbdafe33c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.679174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84072c63-2bd0-420f-8426-cbdafe33c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.679453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84072c63-2bd0-420f-8426-cbdafe33c3e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.680877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84072c63-2bd0-420f-8426-cbdafe33c3e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:58.682504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84072c63-2bd0-420f-8426-cbdafe33c3e9 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:58.689768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 159/306] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:03:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:58.689768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b959-0f9c-4e88-893e-d647c12af1e0 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.689768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b959-0f9c-4e88-893e-d647c12af1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.689768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95b959-0f9c-4e88-893e-d647c12af1e0 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.689768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 160/307] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:03:58.693387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95b959-0f9c-4e88-893e-d647c12af1e0 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:58.782840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24b7f3-5465-4821-87a3-41088f1205de None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:58.801294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95b959-0f9c-4e88-893e-d647c12af1e0 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:58.820836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503502f-18f8-4b34-ad2f-1f718da57d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:58.821437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503502f-18f8-4b34-ad2f-1f718da57d92 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.821717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503502f-18f8-4b34-ad2f-1f718da57d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.822020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503502f-18f8-4b34-ad2f-1f718da57d92 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.822778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503502f-18f8-4b34-ad2f-1f718da57d92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:58.823087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503502f-18f8-4b34-ad2f-1f718da57d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:58.824556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503502f-18f8-4b34-ad2f-1f718da57d92 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:58.833366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 148/308] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:58.834673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571324f-546d-4dc8-9f62-54df68bf9109 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.834842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571324f-546d-4dc8-9f62-54df68bf9109 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.835126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571324f-546d-4dc8-9f62-54df68bf9109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.841284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865393c-3f6e-4165-82bf-185b1265bfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:58.842015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865393c-3f6e-4165-82bf-185b1265bfc6 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:58.842302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865393c-3f6e-4165-82bf-185b1265bfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:58.842582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865393c-3f6e-4165-82bf-185b1265bfc6 None admin] PATH_INFO: `/v3/groups` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:58.843291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865393c-3f6e-4165-82bf-185b1265bfc6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:58.844929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865393c-3f6e-4165-82bf-185b1265bfc6 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:58.903250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f571324f-546d-4dc8-9f62-54df68bf9109 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:03:58.904438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f571324f-546d-4dc8-9f62-54df68bf9109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:03:58.937114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 161/309] 10.222.0.25 () {60 vars in 1134 bytes} [Fri May 13 13:03:58 2022] POST /identity/v3/groups => generated 229 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:03:58.938245 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865393c-3f6e-4165-82bf-185b1265bfc6 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:58.996600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 149/310] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:03:59.015032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571324f-546d-4dc8-9f62-54df68bf9109 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:59.076168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865393c-3f6e-4165-82bf-185b1265bfc6 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:59.112607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc82c9-f35e-43aa-b6a4-eb3763dd904f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:59.113703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc82c9-f35e-43aa-b6a4-eb3763dd904f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:59.113944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc82c9-f35e-43aa-b6a4-eb3763dd904f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:59.114115 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc82c9-f35e-43aa-b6a4-eb3763dd904f None admin] PATH_INFO: `/v3/users/e03e43b1b63e47c691e4fcb557a3ef98` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:59.131997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc82c9-f35e-43aa-b6a4-eb3763dd904f None admin] RBAC: Authorizing `identity:get_user(user_id=e03e43b1b63e47c691e4fcb557a3ef98)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:59.132855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc82c9-f35e-43aa-b6a4-eb3763dd904f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:59.134779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 162/311] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:58 2022] GET /identity/v3/users/e03e43b1b63e47c691e4fcb557a3ef98 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:59.141091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571324f-546d-4dc8-9f62-54df68bf9109 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:59.142765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc82c9-f35e-43aa-b6a4-eb3763dd904f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:59.184350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f010a-227d-4b96-8a17-e0de9318d60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:59.185176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f010a-227d-4b96-8a17-e0de9318d60d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:59.185455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f010a-227d-4b96-8a17-e0de9318d60d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:59.185729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f010a-227d-4b96-8a17-e0de9318d60d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:59.197059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f010a-227d-4b96-8a17-e0de9318d60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-055f010a-227d-4b96-8a17-e0de9318d60d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:03:59.204953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:03:59.206152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:03:59.207098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 150/312] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:03:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:03:59.213325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f010a-227d-4b96-8a17-e0de9318d60d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:59.258563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc82c9-f35e-43aa-b6a4-eb3763dd904f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:59.294366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:59.295129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:59.295489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:59.295839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:59.296749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] RBAC: Adding query filter params (role.id=1163600e955a4a41905c8cc295ce219b, user.id=e03e43b1b63e47c691e4fcb557a3ef98, scope.system=all) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:59.297134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:59.299093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:59.306032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 163/313] 10.222.0.25 () {56 vars in 1308 bytes} [Fri May 13 13:03:59 2022] GET /identity/v3/role_assignments?user.id=e03e43b1b63e47c691e4fcb557a3ef98&scope.system=all&role.id=1163600e955a4a41905c8cc295ce219b => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:59.328428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f010a-227d-4b96-8a17-e0de9318d60d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:59.366331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:59.367359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:59.367791 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:59.368222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:59.369174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:03:59.369633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:59.371818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:59.382189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 151/314] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:03:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:59.388078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:59.502309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3837da5-3e48-4e01-9f62-35f75db715a2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:59.536861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc6423-2fed-4026-a9ee-29a2bc2ba260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:59.537817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6423-2fed-4026-a9ee-29a2bc2ba260 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:59.538113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6423-2fed-4026-a9ee-29a2bc2ba260 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:59.538388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6423-2fed-4026-a9ee-29a2bc2ba260 None admin] PATH_INFO: `/v3/users/bae662443374453981a7ccbb95edc250` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:59.539604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc6423-2fed-4026-a9ee-29a2bc2ba260 None admin] RBAC: Authorizing `identity:get_user(user_id=bae662443374453981a7ccbb95edc250)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:59.541074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc6423-2fed-4026-a9ee-29a2bc2ba260 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:59.543530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 164/315] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:03:59 2022] GET /identity/v3/users/bae662443374453981a7ccbb95edc250 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:03:59.552023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:59.653483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d37e3-8903-4a6d-bead-68bf60abe1e2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:03:59.686960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b4fb2-3a4a-41d6-b149-b2361277dbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:03:59.688005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b4fb2-3a4a-41d6-b149-b2361277dbfe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:03:59.688491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b4fb2-3a4a-41d6-b149-b2361277dbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:03:59.689025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b4fb2-3a4a-41d6-b149-b2361277dbfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:03:59.754729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b4fb2-3a4a-41d6-b149-b2361277dbfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:03:59.756548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b4fb2-3a4a-41d6-b149-b2361277dbfe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:03:59.858811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 152/316] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:03:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:03:59.866865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc6423-2fed-4026-a9ee-29a2bc2ba260 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:03:59.967349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc6423-2fed-4026-a9ee-29a2bc2ba260 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:00.001329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308c174-1662-4d9a-b5ae-93450ef5bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:00.002549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308c174-1662-4d9a-b5ae-93450ef5bc8c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:00.003056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308c174-1662-4d9a-b5ae-93450ef5bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:00.003528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308c174-1662-4d9a-b5ae-93450ef5bc8c None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:00.004962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308c174-1662-4d9a-b5ae-93450ef5bc8c None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:00.006665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308c174-1662-4d9a-b5ae-93450ef5bc8c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:00.008837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 165/317] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:03:59 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:00.014804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b4fb2-3a4a-41d6-b149-b2361277dbfe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:00.116113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b4fb2-3a4a-41d6-b149-b2361277dbfe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:00.151282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080777b6-041c-4b80-8007-ea5edfcdf2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:00.152292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080777b6-041c-4b80-8007-ea5edfcdf2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:00.152777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080777b6-041c-4b80-8007-ea5edfcdf2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:00.153276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080777b6-041c-4b80-8007-ea5edfcdf2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:00.154359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080777b6-041c-4b80-8007-ea5edfcdf2b2 None admin] RBAC: Adding query filter params (role.id=1163600e955a4a41905c8cc295ce219b, scope.project.id=3bd04aa0eebc4c8390eec35a93467757, user.id=bae662443374453981a7ccbb95edc250) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:00.154871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080777b6-041c-4b80-8007-ea5edfcdf2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:00.156517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080777b6-041c-4b80-8007-ea5edfcdf2b2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:00.163230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 153/318] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:00 2022] GET /identity/v3/role_assignments?user.id=bae662443374453981a7ccbb95edc250&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=1163600e955a4a41905c8cc295ce219b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:00.953569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6571dc-6254-4139-bc7f-0b751b011d97 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:00.954095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6571dc-6254-4139-bc7f-0b751b011d97 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:00.954095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6571dc-6254-4139-bc7f-0b751b011d97 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:00.955738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 166/319] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:00.960654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47682e40-c2c9-4bdf-ad91-662e176c9c7a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:00.961047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47682e40-c2c9-4bdf-ad91-662e176c9c7a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:00.961207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47682e40-c2c9-4bdf-ad91-662e176c9c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:00.978152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-47682e40-c2c9-4bdf-ad91-662e176c9c7a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:00.979322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-47682e40-c2c9-4bdf-ad91-662e176c9c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:01.046131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 154/320] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:01.069079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6571dc-6254-4139-bc7f-0b751b011d97 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:01.170463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6571dc-6254-4139-bc7f-0b751b011d97 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:01.203801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358559b0-e81e-4313-8cc2-b05c97448415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:01.204523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358559b0-e81e-4313-8cc2-b05c97448415 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.204804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358559b0-e81e-4313-8cc2-b05c97448415 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.205076 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358559b0-e81e-4313-8cc2-b05c97448415 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.210320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fe9a5-5289-48a9-97e3-ca4f9370bf4a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.210609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fe9a5-5289-48a9-97e3-ca4f9370bf4a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.210887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fe9a5-5289-48a9-97e3-ca4f9370bf4a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.212378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 155/321] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:04:01.215844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358559b0-e81e-4313-8cc2-b05c97448415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:01.217684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3e54-f6b9-4917-bea4-38479d39e83d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.218006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3e54-f6b9-4917-bea4-38479d39e83d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.218282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3e54-f6b9-4917-bea4-38479d39e83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-358559b0-e81e-4313-8cc2-b05c97448415 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:01.223530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:01.224376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:01.225177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 167/322] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:01.230795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358559b0-e81e-4313-8cc2-b05c97448415 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:01.238746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1fbc3e54-f6b9-4917-bea4-38479d39e83d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:01.239914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1fbc3e54-f6b9-4917-bea4-38479d39e83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:01.304885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 156/323] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:01.326915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3e54-f6b9-4917-bea4-38479d39e83d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:01.333501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358559b0-e81e-4313-8cc2-b05c97448415 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:01.369680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:01.370415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.370704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.370984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.371730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:01.372036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:01.373482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:01.381470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 168/324] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:01.386848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:01.430937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3e54-f6b9-4917-bea4-38479d39e83d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:01.469624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489e324-2130-4dcf-8848-563ae2686e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:01.470370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489e324-2130-4dcf-8848-563ae2686e97 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.470653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489e324-2130-4dcf-8848-563ae2686e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.470928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489e324-2130-4dcf-8848-563ae2686e97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.472093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489e324-2130-4dcf-8848-563ae2686e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:01.474883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489e324-2130-4dcf-8848-563ae2686e97 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:01.476882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 157/325] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:01.482985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489e324-2130-4dcf-8848-563ae2686e97 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:01.494376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0a185-e759-4260-be38-5fb4aee4f0a7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:01.528384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e37e-9dc0-4eab-8b24-9f98eadaae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:01.529253 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e37e-9dc0-4eab-8b24-9f98eadaae18 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.529531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e37e-9dc0-4eab-8b24-9f98eadaae18 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.529816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e37e-9dc0-4eab-8b24-9f98eadaae18 None admin] PATH_INFO: `/v3/users/5ec73f7bd93146458ec5bcc804b53ec4` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.530998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34e37e-9dc0-4eab-8b24-9f98eadaae18 None admin] RBAC: Authorizing `identity:get_user(user_id=5ec73f7bd93146458ec5bcc804b53ec4)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:01.532439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34e37e-9dc0-4eab-8b24-9f98eadaae18 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:01.534241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 169/326] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:04:01 2022] GET /identity/v3/users/5ec73f7bd93146458ec5bcc804b53ec4 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:01.539658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e37e-9dc0-4eab-8b24-9f98eadaae18 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:01.587408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489e324-2130-4dcf-8848-563ae2686e97 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:01.620513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37d66b-f409-4fab-a477-13f467c424de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:01.621712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37d66b-f409-4fab-a477-13f467c424de None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.622024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37d66b-f409-4fab-a477-13f467c424de None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.622300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37d66b-f409-4fab-a477-13f467c424de None admin] PATH_INFO: `/v3/groups` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.622982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37d66b-f409-4fab-a477-13f467c424de None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:01.624456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37d66b-f409-4fab-a477-13f467c424de None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:01.644933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e37e-9dc0-4eab-8b24-9f98eadaae18 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:01.676536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e93d5b-1e4c-4808-89d5-d4d6d3ecb3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:01.677184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e93d5b-1e4c-4808-89d5-d4d6d3ecb3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.677470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e93d5b-1e4c-4808-89d5-d4d6d3ecb3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.677749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e93d5b-1e4c-4808-89d5-d4d6d3ecb3b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.678677 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e93d5b-1e4c-4808-89d5-d4d6d3ecb3b8 None admin] RBAC: Adding query filter params (role.id=3552540dc0054d2e87fad160a358c58b, user.id=5ec73f7bd93146458ec5bcc804b53ec4, scope.system=all) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:01.678990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e93d5b-1e4c-4808-89d5-d4d6d3ecb3b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:01.680416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e93d5b-1e4c-4808-89d5-d4d6d3ecb3b8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:01.692104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 170/327] 10.222.0.25 () {56 vars in 1308 bytes} [Fri May 13 13:04:01 2022] GET /identity/v3/role_assignments?user.id=5ec73f7bd93146458ec5bcc804b53ec4&scope.system=all&role.id=3552540dc0054d2e87fad160a358c58b => generated 510 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:01.693296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159a6e4-a446-4a66-bb82-8696b0aeb3f2 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.693590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159a6e4-a446-4a66-bb82-8696b0aeb3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.693899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159a6e4-a446-4a66-bb82-8696b0aeb3f2 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.695134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 171/328] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:01.701042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c832cf-3176-4ae8-9700-20888d7a78f6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.701381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c832cf-3176-4ae8-9700-20888d7a78f6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.701667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c832cf-3176-4ae8-9700-20888d7a78f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:01.719493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b5c832cf-3176-4ae8-9700-20888d7a78f6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:01.720854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b5c832cf-3176-4ae8-9700-20888d7a78f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:01.727942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 158/329] 10.222.0.25 () {60 vars in 1134 bytes} [Fri May 13 13:04:01 2022] POST /identity/v3/groups => generated 226 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:04:01.800832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 172/330] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:01.824694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37d66b-f409-4fab-a477-13f467c424de None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:01.957719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37d66b-f409-4fab-a477-13f467c424de None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:01.993040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0d75-540b-47ee-afef-191b8834f049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:01.993779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0d75-540b-47ee-afef-191b8834f049 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:01.994090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0d75-540b-47ee-afef-191b8834f049 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:01.994365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0d75-540b-47ee-afef-191b8834f049 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:02.006191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d0d75-540b-47ee-afef-191b8834f049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1e7d0d75-540b-47ee-afef-191b8834f049 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:02.013938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:02.014821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:02.015639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 159/331] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:02.022231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c832cf-3176-4ae8-9700-20888d7a78f6 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:02.121575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c832cf-3176-4ae8-9700-20888d7a78f6 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:02.156638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:02.157320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:02.157607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:02.157962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:02.158812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:02.159125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:02.160575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:02.171047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 173/332] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:02.176453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0d75-540b-47ee-afef-191b8834f049 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:02.281256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0d75-540b-47ee-afef-191b8834f049 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:02.316529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550002b3-2b25-4d45-a642-4d9d178ae029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:02.317411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550002b3-2b25-4d45-a642-4d9d178ae029 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:02.317697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550002b3-2b25-4d45-a642-4d9d178ae029 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:02.318003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550002b3-2b25-4d45-a642-4d9d178ae029 None admin] PATH_INFO: `/v3/users/e03e43b1b63e47c691e4fcb557a3ef98` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:02.319231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550002b3-2b25-4d45-a642-4d9d178ae029 None admin] RBAC: Authorizing `identity:get_user(user_id=e03e43b1b63e47c691e4fcb557a3ef98)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:02.320706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550002b3-2b25-4d45-a642-4d9d178ae029 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:02.322800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 160/333] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:04:02 2022] GET /identity/v3/users/e03e43b1b63e47c691e4fcb557a3ef98 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:02.328480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:02.467549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bbd52-57fd-4450-839d-094d756fdd7e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:02.512109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19089f2c-5de1-4cec-8c20-00d23ad5b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:02.512978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089f2c-5de1-4cec-8c20-00d23ad5b692 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:02.513263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089f2c-5de1-4cec-8c20-00d23ad5b692 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:02.513539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089f2c-5de1-4cec-8c20-00d23ad5b692 None admin] PATH_INFO: `/v3/system/users/e03e43b1b63e47c691e4fcb557a3ef98/roles/1163600e955a4a41905c8cc295ce219b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:02.515091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19089f2c-5de1-4cec-8c20-00d23ad5b692 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e03e43b1b63e47c691e4fcb557a3ef98, role_id=1163600e955a4a41905c8cc295ce219b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:02.516486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19089f2c-5de1-4cec-8c20-00d23ad5b692 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:02.591022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 174/334] 10.222.0.25 () {58 vars in 1334 bytes} [Fri May 13 13:04:02 2022] PUT /identity/v3/system/users/e03e43b1b63e47c691e4fcb557a3ef98/roles/1163600e955a4a41905c8cc295ce219b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:04:04.379647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac571b-d999-4db5-a881-e784affb3005 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:04.380228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac571b-d999-4db5-a881-e784affb3005 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:04.380228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac571b-d999-4db5-a881-e784affb3005 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:04.381835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 161/335] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:04.387020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d2fe-f884-416f-89ac-fbbfd0a5b818 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:04.387442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d2fe-f884-416f-89ac-fbbfd0a5b818 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:04.387617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d2fe-f884-416f-89ac-fbbfd0a5b818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:04.408695 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-59c4d2fe-f884-416f-89ac-fbbfd0a5b818 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:04.409920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-59c4d2fe-f884-416f-89ac-fbbfd0a5b818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:04.505812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 175/336] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:04.539271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac571b-d999-4db5-a881-e784affb3005 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:04.665667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac571b-d999-4db5-a881-e784affb3005 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:04.702578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fed651-4285-4e35-9829-77ebd784cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:04.703425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fed651-4285-4e35-9829-77ebd784cda5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:04.703707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fed651-4285-4e35-9829-77ebd784cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:04.703984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fed651-4285-4e35-9829-77ebd784cda5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:04.719147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fed651-4285-4e35-9829-77ebd784cda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-80fed651-4285-4e35-9829-77ebd784cda5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:04.727557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:04.728548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:04.729366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 162/337] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:04:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:04.735466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4d2fe-f884-416f-89ac-fbbfd0a5b818 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:04.850708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa7e6d-a523-4e90-a0e6-ec03eb7d9443 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:04.851056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa7e6d-a523-4e90-a0e6-ec03eb7d9443 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:04.851671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa7e6d-a523-4e90-a0e6-ec03eb7d9443 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:04.853344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 163/338] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:04.860778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81648c4b-90ec-4028-bcf5-44e89db3c3e4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:04.861069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81648c4b-90ec-4028-bcf5-44e89db3c3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:04.861340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81648c4b-90ec-4028-bcf5-44e89db3c3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:04.868148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4d2fe-f884-416f-89ac-fbbfd0a5b818 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:04.886305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-81648c4b-90ec-4028-bcf5-44e89db3c3e4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:04.887502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-81648c4b-90ec-4028-bcf5-44e89db3c3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:04.907445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:04.908478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:04.908980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:04.909465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:04.910459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:04.910996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:04.912594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:04.922350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 176/339] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:04.927570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:05.017920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 164/340] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:05.069114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81648c4b-90ec-4028-bcf5-44e89db3c3e4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:05.102508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ad361-d710-4f32-ad3d-726be438f63c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:05.181877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217021b-7be0-4cd1-9b18-1036ebb15e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:05.182447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217021b-7be0-4cd1-9b18-1036ebb15e19 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:05.183277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217021b-7be0-4cd1-9b18-1036ebb15e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:05.183553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217021b-7be0-4cd1-9b18-1036ebb15e19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:05.256189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81648c4b-90ec-4028-bcf5-44e89db3c3e4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:05.290305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217021b-7be0-4cd1-9b18-1036ebb15e19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:05.291780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217021b-7be0-4cd1-9b18-1036ebb15e19 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:05.321882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bad5f9-d40c-4793-9495-2d024d353930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:05.322684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bad5f9-d40c-4793-9495-2d024d353930 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:05.322969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bad5f9-d40c-4793-9495-2d024d353930 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:05.323760 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bad5f9-d40c-4793-9495-2d024d353930 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:05.341507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bad5f9-d40c-4793-9495-2d024d353930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-00bad5f9-d40c-4793-9495-2d024d353930 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:05.349230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:05.350156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:05.350963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:05.351652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 165/341] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:05.352216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7ce1-9c56-483e-bcfb-56c5e71c443c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:05.354808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7ce1-9c56-483e-bcfb-56c5e71c443c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:05.355816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7ce1-9c56-483e-bcfb-56c5e71c443c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:05.363291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 166/342] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:05.365124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b7ce1-9c56-483e-bcfb-56c5e71c443c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:05.452104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 177/343] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:04:05.457660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1fd7c-ea13-4609-b922-d4a0504eb89c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:05.457981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1fd7c-ea13-4609-b922-d4a0504eb89c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:05.458260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1fd7c-ea13-4609-b922-d4a0504eb89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:05.482003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2fd1fd7c-ea13-4609-b922-d4a0504eb89c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:05.482746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2fd1fd7c-ea13-4609-b922-d4a0504eb89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:05.596203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b7ce1-9c56-483e-bcfb-56c5e71c443c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:05.642086 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 178/344] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:05.643028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1fd7c-ea13-4609-b922-d4a0504eb89c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:05.666530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:05.667784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:05.668073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:05.668355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:05.669120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:05.669433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:05.670861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:05.679989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 167/345] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:05.685909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:05.751317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1fd7c-ea13-4609-b922-d4a0504eb89c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:05.785407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8162aa5-871b-4bbe-8159-a17e420c4aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:05.786299 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8162aa5-871b-4bbe-8159-a17e420c4aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:05.786579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8162aa5-871b-4bbe-8159-a17e420c4aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:05.786850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8162aa5-871b-4bbe-8159-a17e420c4aa0 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:05.788042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8162aa5-871b-4bbe-8159-a17e420c4aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:05.789613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8162aa5-871b-4bbe-8159-a17e420c4aa0 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:05.795869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 179/346] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:05 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:05.797016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8162aa5-871b-4bbe-8159-a17e420c4aa0 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:05.821845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423223f-8110-4d47-9af6-e72d756bb2e8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:05.880888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9575d-25c8-44ad-af39-63f3c24ce3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:05.881577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9575d-25c8-44ad-af39-63f3c24ce3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:05.881899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9575d-25c8-44ad-af39-63f3c24ce3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:05.882182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9575d-25c8-44ad-af39-63f3c24ce3c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:05.912273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8162aa5-871b-4bbe-8159-a17e420c4aa0 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:05.965579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebdd42-a68b-4566-89d3-0b89538ebc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:05.966647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebdd42-a68b-4566-89d3-0b89538ebc6a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:05.967127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebdd42-a68b-4566-89d3-0b89538ebc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:05.967562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebdd42-a68b-4566-89d3-0b89538ebc6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:05.978557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ebdd42-a68b-4566-89d3-0b89538ebc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-d9ebdd42-a68b-4566-89d3-0b89538ebc6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:05.987262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:05.988390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:05.989391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:05.990126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9575d-25c8-44ad-af39-63f3c24ce3c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:05.991181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9575d-25c8-44ad-af39-63f3c24ce3c7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:05.992024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 180/347] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:05.993097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebdd42-a68b-4566-89d3-0b89538ebc6a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:06.114681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebdd42-a68b-4566-89d3-0b89538ebc6a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:06.153088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615c165-8e1a-435c-809c-2044a0c771e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:06.154243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615c165-8e1a-435c-809c-2044a0c771e8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.154679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615c165-8e1a-435c-809c-2044a0c771e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.155150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615c165-8e1a-435c-809c-2044a0c771e8 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.161950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615c165-8e1a-435c-809c-2044a0c771e8 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:06.163986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615c165-8e1a-435c-809c-2044a0c771e8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:06.171688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 181/348] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:05 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:06.172812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615c165-8e1a-435c-809c-2044a0c771e8 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:06.225961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 168/349] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:04:06.226875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9575d-25c8-44ad-af39-63f3c24ce3c7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:06.318651 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615c165-8e1a-435c-809c-2044a0c771e8 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:06.358786 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9575d-25c8-44ad-af39-63f3c24ce3c7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:06.366759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:06.367402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.367687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.367960 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.368713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:06.369019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:06.371189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:06.384555 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 182/350] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:06.384921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:06.402582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9d1df-5134-4753-9e34-cc81be87dc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:06.403490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9d1df-5134-4753-9e34-cc81be87dc8e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.403921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9d1df-5134-4753-9e34-cc81be87dc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.404360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9d1df-5134-4753-9e34-cc81be87dc8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.405338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9d1df-5134-4753-9e34-cc81be87dc8e None admin] RBAC: Adding query filter params (group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=1eafe2a57502441faa06c7ab6de43b01) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:06.405868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9d1df-5134-4753-9e34-cc81be87dc8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:06.407401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9d1df-5134-4753-9e34-cc81be87dc8e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:06.413720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 169/351] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:06 2022] GET /identity/v3/role_assignments?group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=9065c36fc305418baa5b977858b445dd => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:06.415048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace1506-9ecd-42b5-80e1-fedb3903e744 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.415343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace1506-9ecd-42b5-80e1-fedb3903e744 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.415628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace1506-9ecd-42b5-80e1-fedb3903e744 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.417674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 170/352] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:06.424032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab5229-57e3-4faf-a993-f00f4b2558c2 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.424032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab5229-57e3-4faf-a993-f00f4b2558c2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.424032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab5229-57e3-4faf-a993-f00f4b2558c2 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.425255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 171/353] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:06.429962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab5229-57e3-4faf-a993-f00f4b2558c2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:06.524906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ee7bd-2f10-41e0-ac9b-57c0381f8417 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:06.565598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab5229-57e3-4faf-a993-f00f4b2558c2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:06.575064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa83a8c-a1fb-4dc9-97db-608a79c37469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:06.575986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa83a8c-a1fb-4dc9-97db-608a79c37469 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.576636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa83a8c-a1fb-4dc9-97db-608a79c37469 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.576923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa83a8c-a1fb-4dc9-97db-608a79c37469 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.578871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa83a8c-a1fb-4dc9-97db-608a79c37469 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:06.580569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa83a8c-a1fb-4dc9-97db-608a79c37469 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:06.583257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 183/354] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:06 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:06.584571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd11d6-0297-4411-ac18-a43c991bfe58 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.584867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd11d6-0297-4411-ac18-a43c991bfe58 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.585148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd11d6-0297-4411-ac18-a43c991bfe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.606502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8ac96-629f-452f-b627-067c36603bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:06.607044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-16bd11d6-0297-4411-ac18-a43c991bfe58 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:06.607124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8ac96-629f-452f-b627-067c36603bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.607406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8ac96-629f-452f-b627-067c36603bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.607680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8ac96-629f-452f-b627-067c36603bd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.607850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-16bd11d6-0297-4411-ac18-a43c991bfe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:06.693077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 184/355] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:06.693639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450836d-6f0a-4188-ba67-e5493b75fc10 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.694016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450836d-6f0a-4188-ba67-e5493b75fc10 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.694297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450836d-6f0a-4188-ba67-e5493b75fc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.707313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8ac96-629f-452f-b627-067c36603bd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:06.708793 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8ac96-629f-452f-b627-067c36603bd9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:06.712716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b450836d-6f0a-4188-ba67-e5493b75fc10 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:06.714764 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b450836d-6f0a-4188-ba67-e5493b75fc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:06.795996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 185/356] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:06.797463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08796291-7907-4f12-ae63-aa1f28bbb860 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.797757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08796291-7907-4f12-ae63-aa1f28bbb860 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.798063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08796291-7907-4f12-ae63-aa1f28bbb860 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.799463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 186/357] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:06.800183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08796291-7907-4f12-ae63-aa1f28bbb860 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:06.831691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 172/358] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:06.831691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8ac96-629f-452f-b627-067c36603bd9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:06.929837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08796291-7907-4f12-ae63-aa1f28bbb860 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:06.945245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8ac96-629f-452f-b627-067c36603bd9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:06.969072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23408ef2-ec1c-4b35-bfb2-294f8f13e4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:06.969970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23408ef2-ec1c-4b35-bfb2-294f8f13e4bc None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.970254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23408ef2-ec1c-4b35-bfb2-294f8f13e4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.970530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23408ef2-ec1c-4b35-bfb2-294f8f13e4bc None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.971742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23408ef2-ec1c-4b35-bfb2-294f8f13e4bc None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:06.973210 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23408ef2-ec1c-4b35-bfb2-294f8f13e4bc None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:06.975073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 187/359] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:06 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:06.978031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25037-4b48-404d-990e-9bdb2c9b2a4c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.978322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25037-4b48-404d-990e-9bdb2c9b2a4c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.978605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25037-4b48-404d-990e-9bdb2c9b2a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.988223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c9167-dc06-4b35-8fae-0b42e8347fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:06.989311 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c9167-dc06-4b35-8fae-0b42e8347fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:06.989879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c9167-dc06-4b35-8fae-0b42e8347fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:06.990657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c9167-dc06-4b35-8fae-0b42e8347fc4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:06.998155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-8dd25037-4b48-404d-990e-9bdb2c9b2a4c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:06.999545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8dd25037-4b48-404d-990e-9bdb2c9b2a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:07.004871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c9167-dc06-4b35-8fae-0b42e8347fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-360c9167-dc06-4b35-8fae-0b42e8347fc4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:07.012956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:07.013873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 13 13:04:07.014665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:07.021973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 173/360] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:07.023090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c9167-dc06-4b35-8fae-0b42e8347fc4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.080942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 188/361] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:07.082164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd25037-4b48-404d-990e-9bdb2c9b2a4c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.152286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c9167-dc06-4b35-8fae-0b42e8347fc4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.194536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8b468-e7df-4a25-bc2b-ecb605710579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.195911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8b468-e7df-4a25-bc2b-ecb605710579 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.196351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8b468-e7df-4a25-bc2b-ecb605710579 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.196637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8b468-e7df-4a25-bc2b-ecb605710579 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.207292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8b468-e7df-4a25-bc2b-ecb605710579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:07.207941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd25037-4b48-404d-990e-9bdb2c9b2a4c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-63c8b468-e7df-4a25-bc2b-ecb605710579 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:07.215834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:07.216732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:07.217531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 174/362] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:07.218239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8b468-e7df-4a25-bc2b-ecb605710579 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.260691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc0ad0-0a00-4cee-bfa4-2e28ecc477cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.263779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc0ad0-0a00-4cee-bfa4-2e28ecc477cd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.263779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc0ad0-0a00-4cee-bfa4-2e28ecc477cd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.263779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc0ad0-0a00-4cee-bfa4-2e28ecc477cd None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.265071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc0ad0-0a00-4cee-bfa4-2e28ecc477cd None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:07.266578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc0ad0-0a00-4cee-bfa4-2e28ecc477cd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:07.268293 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 189/363] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:07.269366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc0ad0-0a00-4cee-bfa4-2e28ecc477cd None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.355872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8b468-e7df-4a25-bc2b-ecb605710579 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.390898 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.391555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.391835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.392107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.393534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] RBAC: Adding query filter params (group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=1eafe2a57502441faa06c7ab6de43b01) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:07.393534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:07.397540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:07.400330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc0ad0-0a00-4cee-bfa4-2e28ecc477cd None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.400449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 175/364] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/role_assignments?group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=3552540dc0054d2e87fad160a358c58b => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:07.405857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.440596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.441417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.441707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.442035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.442798 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:07.443108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:07.444592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:07.455749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 190/365] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:07.456570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.508393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1fd0e-81f5-42c7-acea-20367a267bfa None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.546025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5e32d-4a7f-4938-9d53-469f142cef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.546844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5e32d-4a7f-4938-9d53-469f142cef00 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.547127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5e32d-4a7f-4938-9d53-469f142cef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.547400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5e32d-4a7f-4938-9d53-469f142cef00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.558755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5e32d-4a7f-4938-9d53-469f142cef00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2fd5e32d-4a7f-4938-9d53-469f142cef00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:07.568044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:07.568999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:07.569812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 176/366] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:07.570475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5e32d-4a7f-4938-9d53-469f142cef00 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.576872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9d02b-b615-4854-8299-631d29ac2da5 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.614824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.615521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.615808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.616086 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.616845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:07.617154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:07.618605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:07.630823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 191/367] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:07.631126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.700757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5e32d-4a7f-4938-9d53-469f142cef00 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.741671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fe63-8db0-493b-93ce-53e8532b4f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.742594 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fe63-8db0-493b-93ce-53e8532b4f50 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.742876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fe63-8db0-493b-93ce-53e8532b4f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.743153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07fe63-8db0-493b-93ce-53e8532b4f50 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.744616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07fe63-8db0-493b-93ce-53e8532b4f50 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:07.746297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07fe63-8db0-493b-93ce-53e8532b4f50 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:07.748974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 177/368] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:07.750363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fe63-8db0-493b-93ce-53e8532b4f50 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.758315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bdea32-cc6d-4bf7-99dc-d20f79547a36 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.795900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba9c2c-01f8-4ceb-8223-4e614f5fa8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.796791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba9c2c-01f8-4ceb-8223-4e614f5fa8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.797078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba9c2c-01f8-4ceb-8223-4e614f5fa8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.797361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba9c2c-01f8-4ceb-8223-4e614f5fa8c7 None admin] PATH_INFO: `/v3/users/e03e43b1b63e47c691e4fcb557a3ef98` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.798617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba9c2c-01f8-4ceb-8223-4e614f5fa8c7 None admin] RBAC: Authorizing `identity:get_user(user_id=e03e43b1b63e47c691e4fcb557a3ef98)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:07.800067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba9c2c-01f8-4ceb-8223-4e614f5fa8c7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:07.801703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 192/369] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/users/e03e43b1b63e47c691e4fcb557a3ef98 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:07.802504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba9c2c-01f8-4ceb-8223-4e614f5fa8c7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.871304 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07fe63-8db0-493b-93ce-53e8532b4f50 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.911854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.912528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.912816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.913095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:07.913890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:07.914205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:07.915581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:07.924926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 178/370] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:07.925570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:07.928438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba9c2c-01f8-4ceb-8223-4e614f5fa8c7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:07.969928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddf5ec-db95-430e-814c-56b21688dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:07.971370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddf5ec-db95-430e-814c-56b21688dafe None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:07.971662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddf5ec-db95-430e-814c-56b21688dafe None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:07.971941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddf5ec-db95-430e-814c-56b21688dafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:08.043393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1de3-e09e-4f09-81bc-c058ef388a2f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:08.053704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddf5ec-db95-430e-814c-56b21688dafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:08.055775 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddf5ec-db95-430e-814c-56b21688dafe None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:08.083406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:08.084066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:08.084346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:08.084623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:08.085460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] RBAC: Adding query filter params (group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=3bd04aa0eebc4c8390eec35a93467757) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:08.085773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:08.087240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:08.095375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 179/371] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/role_assignments?group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=3552540dc0054d2e87fad160a358c58b => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:08.096210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:08.231523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96addb-b075-4f0d-aa30-06a0da17b51e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:08.258715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 193/372] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:08.259586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddf5ec-db95-430e-814c-56b21688dafe None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:08.272915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:08.273632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:08.273945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:08.274225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:08.275076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] RBAC: Adding query filter params (role.id=1163600e955a4a41905c8cc295ce219b, user.id=e03e43b1b63e47c691e4fcb557a3ef98, scope.system=all) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:08.275392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:08.276919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:08.290235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 180/373] 10.222.0.25 () {56 vars in 1308 bytes} [Fri May 13 13:04:08 2022] GET /identity/v3/role_assignments?user.id=e03e43b1b63e47c691e4fcb557a3ef98&scope.system=all&role.id=1163600e955a4a41905c8cc295ce219b => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:08.291111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:08.408027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13693fba-7021-4eaa-814d-c2d75f9b369b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:08.451067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8338f-003b-4486-91f8-b2053af79c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:08.451956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8338f-003b-4486-91f8-b2053af79c45 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:08.452238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8338f-003b-4486-91f8-b2053af79c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:08.452518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8338f-003b-4486-91f8-b2053af79c45 None admin] PATH_INFO: `/v3/projects/989de7565a074e299634e689ed15d9f9` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:08.454531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8338f-003b-4486-91f8-b2053af79c45 None admin] RBAC: Authorizing `identity:get_project(project_id=989de7565a074e299634e689ed15d9f9)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:08.456014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8338f-003b-4486-91f8-b2053af79c45 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:08.458045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 181/374] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:08 2022] GET /identity/v3/projects/989de7565a074e299634e689ed15d9f9 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:08.467256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8338f-003b-4486-91f8-b2053af79c45 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:08.503371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddf5ec-db95-430e-814c-56b21688dafe None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:08.576152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b450b7f-ec42-45f9-bbd1-8c566be63672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:08.576818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b450b7f-ec42-45f9-bbd1-8c566be63672 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:08.577104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b450b7f-ec42-45f9-bbd1-8c566be63672 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:08.577623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b450b7f-ec42-45f9-bbd1-8c566be63672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:08.671085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8338f-003b-4486-91f8-b2053af79c45 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:08.690685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b450b7f-ec42-45f9-bbd1-8c566be63672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:08.692187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b450b7f-ec42-45f9-bbd1-8c566be63672 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:08.722179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e28ae-1fb9-4a99-bdf4-4ee1bfec9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:08.723873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e28ae-1fb9-4a99-bdf4-4ee1bfec9a6b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:08.724395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e28ae-1fb9-4a99-bdf4-4ee1bfec9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:08.724969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e28ae-1fb9-4a99-bdf4-4ee1bfec9a6b None admin] PATH_INFO: `/v3/groups/4ff4dc1e9557499db5d4c29b994a0cc5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:08.734322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e28ae-1fb9-4a99-bdf4-4ee1bfec9a6b None admin] RBAC: Authorizing `identity:get_group(group_id=4ff4dc1e9557499db5d4c29b994a0cc5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:08.736149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e28ae-1fb9-4a99-bdf4-4ee1bfec9a6b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:08.738451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 182/375] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:08 2022] GET /identity/v3/groups/4ff4dc1e9557499db5d4c29b994a0cc5 => generated 229 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:08.744512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e28ae-1fb9-4a99-bdf4-4ee1bfec9a6b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:08.867162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e28ae-1fb9-4a99-bdf4-4ee1bfec9a6b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:08.906519 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:08.907575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:08.908087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:08.908619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:08.909716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] RBAC: Adding query filter params (group.id=4ff4dc1e9557499db5d4c29b994a0cc5, role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=989de7565a074e299634e689ed15d9f9) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:08.910310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:08.912875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:08.915154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 194/376] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:08.920147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 183/377] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:08 2022] GET /identity/v3/role_assignments?group.id=4ff4dc1e9557499db5d4c29b994a0cc5&scope.project.id=989de7565a074e299634e689ed15d9f9&role.id=663341777b0b44128ed78fe8d40f6462 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:08.920932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b450b7f-ec42-45f9-bbd1-8c566be63672 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:09.052274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b450b7f-ec42-45f9-bbd1-8c566be63672 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:09.098974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bce110-85df-4398-a96c-18f5a475aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:09.099855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce110-85df-4398-a96c-18f5a475aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:09.100139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce110-85df-4398-a96c-18f5a475aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:09.100413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce110-85df-4398-a96c-18f5a475aaf1 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:09.102599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bce110-85df-4398-a96c-18f5a475aaf1 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:09.104095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bce110-85df-4398-a96c-18f5a475aaf1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:09.106813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 195/378] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:08 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:09.112797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:09.158465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b959262-bf8f-4799-92aa-beada5e71aa1 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:09.158788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b959262-bf8f-4799-92aa-beada5e71aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:09.159037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b959262-bf8f-4799-92aa-beada5e71aa1 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:09.160653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 196/379] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:09.165880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d793-743f-4c56-b227-f7ca8cecfc04 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:09.166172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d793-743f-4c56-b227-f7ca8cecfc04 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:09.166562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61d793-743f-4c56-b227-f7ca8cecfc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:09.191739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-db61d793-743f-4c56-b227-f7ca8cecfc04 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:09.192982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-db61d793-743f-4c56-b227-f7ca8cecfc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:09.251290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb6061-1468-46bd-abf6-9a10208045c7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:09.284412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 197/380] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:09.294254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b7f4e-cf19-46d3-866e-1ad40fe706d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:09.295432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b7f4e-cf19-46d3-866e-1ad40fe706d5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:09.295937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b7f4e-cf19-46d3-866e-1ad40fe706d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:09.296463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b7f4e-cf19-46d3-866e-1ad40fe706d5 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:09.297908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b7f4e-cf19-46d3-866e-1ad40fe706d5 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:09.299688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b7f4e-cf19-46d3-866e-1ad40fe706d5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:09.301886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 184/381] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:09 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:09.308645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d793-743f-4c56-b227-f7ca8cecfc04 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:09.321708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b7f4e-cf19-46d3-866e-1ad40fe706d5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:09.417190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61d793-743f-4c56-b227-f7ca8cecfc04 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:09.452617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:09.453284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:09.453572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:09.453879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:09.454721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] RBAC: Adding query filter params (group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=3bd04aa0eebc4c8390eec35a93467757) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:09.455031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:09.456549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:09.462219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 198/382] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:09 2022] GET /identity/v3/role_assignments?group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=9065c36fc305418baa5b977858b445dd => generated 260 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:09.463909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b7f4e-cf19-46d3-866e-1ad40fe706d5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:09.506931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82f8a-260e-4220-bb9b-708d42afdbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:09.507956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82f8a-260e-4220-bb9b-708d42afdbdf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:09.508342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82f8a-260e-4220-bb9b-708d42afdbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:09.508720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82f8a-260e-4220-bb9b-708d42afdbdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:09.521364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee82f8a-260e-4220-bb9b-708d42afdbdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-dee82f8a-260e-4220-bb9b-708d42afdbdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:09.529757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:09.530762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:09.531627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 185/383] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:04:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:09.536982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:09.650334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1b4b-097c-4b2b-afc7-3160af2ba352 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:09.697275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:09.697275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:09.697275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:09.697617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:09.698259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:09.698565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:09.699945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:09.712474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 199/384] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:09.718886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82f8a-260e-4220-bb9b-708d42afdbdf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:09.845610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82f8a-260e-4220-bb9b-708d42afdbdf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:09.888074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bf695-4474-495b-bb05-bf7f22f4b4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:09.889034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bf695-4474-495b-bb05-bf7f22f4b4fc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:09.889317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bf695-4474-495b-bb05-bf7f22f4b4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:09.889592 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bf695-4474-495b-bb05-bf7f22f4b4fc None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:09.890821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bf695-4474-495b-bb05-bf7f22f4b4fc None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:09.892257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bf695-4474-495b-bb05-bf7f22f4b4fc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:09.894218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 186/385] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:09 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:09.899601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:10.013675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9805f74-97b7-4731-88ac-02f64d0fcee2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:10.047253 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d747f-0bf3-4aa0-a7fe-a9a4cb31edb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:10.048116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d747f-0bf3-4aa0-a7fe-a9a4cb31edb6 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:10.048396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d747f-0bf3-4aa0-a7fe-a9a4cb31edb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:10.048671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d747f-0bf3-4aa0-a7fe-a9a4cb31edb6 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:10.049901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d747f-0bf3-4aa0-a7fe-a9a4cb31edb6 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:10.051333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d747f-0bf3-4aa0-a7fe-a9a4cb31edb6 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:10.053330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 200/386] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:09 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:10.059103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bf695-4474-495b-bb05-bf7f22f4b4fc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:10.178519 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bf695-4474-495b-bb05-bf7f22f4b4fc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:10.220648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252cf3b-0c35-4ff4-9458-2bd93c2f6605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:10.221541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252cf3b-0c35-4ff4-9458-2bd93c2f6605 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:10.221884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252cf3b-0c35-4ff4-9458-2bd93c2f6605 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:10.222123 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252cf3b-0c35-4ff4-9458-2bd93c2f6605 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920/roles/9065c36fc305418baa5b977858b445dd` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:10.224358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252cf3b-0c35-4ff4-9458-2bd93c2f6605 None admin] RBAC: Authorizing `identity:create_grant(project_id=1eafe2a57502441faa06c7ab6de43b01, group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role_id=9065c36fc305418baa5b977858b445dd)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:10.225815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252cf3b-0c35-4ff4-9458-2bd93c2f6605 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:10.310320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 187/387] 10.222.0.25 () {58 vars in 1442 bytes} [Fri May 13 13:04:10 2022] PUT /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920/roles/9065c36fc305418baa5b977858b445dd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:04:10.679825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf937-f626-4774-b3db-5a70bc48ec05 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:10.680305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf937-f626-4774-b3db-5a70bc48ec05 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:10.680305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cf937-f626-4774-b3db-5a70bc48ec05 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:10.681943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 201/388] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:10.687538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e05ade-76bd-4e1b-8353-40b17d7f79b9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:10.687838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e05ade-76bd-4e1b-8353-40b17d7f79b9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:10.688129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e05ade-76bd-4e1b-8353-40b17d7f79b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:10.710872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e9e05ade-76bd-4e1b-8353-40b17d7f79b9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:10.712083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e9e05ade-76bd-4e1b-8353-40b17d7f79b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:10.780391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 188/389] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:10.815487 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cf937-f626-4774-b3db-5a70bc48ec05 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:10.944127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cf937-f626-4774-b3db-5a70bc48ec05 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:10.990098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abbb60-43a7-42dc-a565-e8ede9eb3515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:10.990905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abbb60-43a7-42dc-a565-e8ede9eb3515 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:10.991187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abbb60-43a7-42dc-a565-e8ede9eb3515 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:10.991460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abbb60-43a7-42dc-a565-e8ede9eb3515 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.006504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abbb60-43a7-42dc-a565-e8ede9eb3515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-33abbb60-43a7-42dc-a565-e8ede9eb3515 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:11.018699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:11.019838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:11.020844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 202/390] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:11.027389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e05ade-76bd-4e1b-8353-40b17d7f79b9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:11.145671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b54ba-e7fe-4fc9-a0ca-08b39d416abd None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.146040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b54ba-e7fe-4fc9-a0ca-08b39d416abd None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.146274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b54ba-e7fe-4fc9-a0ca-08b39d416abd None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.147888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 203/391] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:11.152876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e05ade-76bd-4e1b-8353-40b17d7f79b9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:11.153091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b8d82-8b43-42be-acc9-08fdc43d1a40 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.153261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b8d82-8b43-42be-acc9-08fdc43d1a40 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.153505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b8d82-8b43-42be-acc9-08fdc43d1a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.193985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:11.194729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.194959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.195239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.196002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:11.196308 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:11.197693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:11.206936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-716b8d82-8b43-42be-acc9-08fdc43d1a40 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:11.207405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 189/392] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:11.208144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-716b8d82-8b43-42be-acc9-08fdc43d1a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:11.213332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:11.292728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 204/393] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:11.320352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39bbd5-fafb-40b6-8ac7-5fbfb3de7e2d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:11.332156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b8d82-8b43-42be-acc9-08fdc43d1a40 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:11.357963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb833e-3616-49b9-a3c1-88dd9c30a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:11.357963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb833e-3616-49b9-a3c1-88dd9c30a187 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.357963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb833e-3616-49b9-a3c1-88dd9c30a187 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.357963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb833e-3616-49b9-a3c1-88dd9c30a187 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.358358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb833e-3616-49b9-a3c1-88dd9c30a187 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:11.365295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb833e-3616-49b9-a3c1-88dd9c30a187 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:11.366978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 190/394] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:11 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:11.374717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb833e-3616-49b9-a3c1-88dd9c30a187 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:11.449055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b8d82-8b43-42be-acc9-08fdc43d1a40 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:11.488893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c092d-4ff0-46fb-997c-5be591ad27c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:11.489694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c092d-4ff0-46fb-997c-5be591ad27c7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.490079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c092d-4ff0-46fb-997c-5be591ad27c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.490358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c092d-4ff0-46fb-997c-5be591ad27c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.521911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c092d-4ff0-46fb-997c-5be591ad27c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-359c092d-4ff0-46fb-997c-5be591ad27c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:11.530171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:11.531084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:11.531871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:11.544684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 205/395] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:11.547553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb833e-3616-49b9-a3c1-88dd9c30a187 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:11.551098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c092d-4ff0-46fb-997c-5be591ad27c7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:11.585008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d3857-5c2f-4ee9-9862-ea5737e849c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:11.585898 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3857-5c2f-4ee9-9862-ea5737e849c7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.586188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3857-5c2f-4ee9-9862-ea5737e849c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.586481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d3857-5c2f-4ee9-9862-ea5737e849c7 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.592120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d3857-5c2f-4ee9-9862-ea5737e849c7 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:11.592120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d3857-5c2f-4ee9-9862-ea5737e849c7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:11.592120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 191/396] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:11 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:11.599057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d3857-5c2f-4ee9-9862-ea5737e849c7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:11.729038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d3857-5c2f-4ee9-9862-ea5737e849c7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:11.754840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c092d-4ff0-46fb-997c-5be591ad27c7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:11.765953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1402793-c905-40ed-ba9c-d41c2edfc49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:11.770388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1402793-c905-40ed-ba9c-d41c2edfc49f None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.770388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1402793-c905-40ed-ba9c-d41c2edfc49f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.770388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1402793-c905-40ed-ba9c-d41c2edfc49f None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.770388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1402793-c905-40ed-ba9c-d41c2edfc49f None admin] RBAC: Authorizing `identity:create_grant(project_id=1eafe2a57502441faa06c7ab6de43b01, group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:11.770855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1402793-c905-40ed-ba9c-d41c2edfc49f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:11.790145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5979c-927a-46be-bf56-cc355b1a4889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:11.791121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5979c-927a-46be-bf56-cc355b1a4889 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.791611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5979c-927a-46be-bf56-cc355b1a4889 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.792630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5979c-927a-46be-bf56-cc355b1a4889 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.793611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5979c-927a-46be-bf56-cc355b1a4889 None admin] RBAC: Adding query filter params (name=member) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:11.794112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5979c-927a-46be-bf56-cc355b1a4889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:11.795691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5979c-927a-46be-bf56-cc355b1a4889 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:11.804311 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 206/397] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:11.805385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b58c6-7910-4eb9-af7d-06235ef63ac5 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.809907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b58c6-7910-4eb9-af7d-06235ef63ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.809907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b58c6-7910-4eb9-af7d-06235ef63ac5 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.809907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 207/398] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:11.814589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b58c6-7910-4eb9-af7d-06235ef63ac5 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:11.882435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 192/399] 10.222.0.25 () {58 vars in 1442 bytes} [Fri May 13 13:04:11 2022] PUT /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:04:11.884331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563b7a3-620c-45ae-9143-abe8a9857c34 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:11.884628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563b7a3-620c-45ae-9143-abe8a9857c34 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:11.884911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563b7a3-620c-45ae-9143-abe8a9857c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:11.914401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4563b7a3-620c-45ae-9143-abe8a9857c34 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:11.915637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4563b7a3-620c-45ae-9143-abe8a9857c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:11.956492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b58c6-7910-4eb9-af7d-06235ef63ac5 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:12.037522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa2d5-c0e0-4c99-b49e-77c10183c0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:12.038823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa2d5-c0e0-4c99-b49e-77c10183c0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.041493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa2d5-c0e0-4c99-b49e-77c10183c0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.042124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa2d5-c0e0-4c99-b49e-77c10183c0c5 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.044391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa2d5-c0e0-4c99-b49e-77c10183c0c5 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:12.046735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa2d5-c0e0-4c99-b49e-77c10183c0c5 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:12.054141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 208/400] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:11 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:12.059197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa2d5-c0e0-4c99-b49e-77c10183c0c5 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:12.080933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 193/401] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:12.106177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563b7a3-620c-45ae-9143-abe8a9857c34 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:12.199606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa2d5-c0e0-4c99-b49e-77c10183c0c5 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:12.238530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563b7a3-620c-45ae-9143-abe8a9857c34 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:12.253278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7a6f0-4953-4636-8161-e4893145a395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:12.254497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a6f0-4953-4636-8161-e4893145a395 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.255075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a6f0-4953-4636-8161-e4893145a395 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.255548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a6f0-4953-4636-8161-e4893145a395 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.262593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7a6f0-4953-4636-8161-e4893145a395 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:12.264354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7a6f0-4953-4636-8161-e4893145a395 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:12.266648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 209/402] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:12 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:12.272348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7a6f0-4953-4636-8161-e4893145a395 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:12.284684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eece4-a11a-4f8a-8d05-79c3dba611fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:12.285503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eece4-a11a-4f8a-8d05-79c3dba611fa None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.285818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eece4-a11a-4f8a-8d05-79c3dba611fa None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.286078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eece4-a11a-4f8a-8d05-79c3dba611fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.305242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888eece4-a11a-4f8a-8d05-79c3dba611fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-888eece4-a11a-4f8a-8d05-79c3dba611fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:12.321093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:12.322035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:12.322855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:12.326516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 194/403] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:12.332086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eece4-a11a-4f8a-8d05-79c3dba611fa None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:12.424491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7a6f0-4953-4636-8161-e4893145a395 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:12.460354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eece4-a11a-4f8a-8d05-79c3dba611fa None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:12.470072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d0d51-12cf-4adb-9c93-67bdc19e0a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:12.471519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d0d51-12cf-4adb-9c93-67bdc19e0a23 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.472265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d0d51-12cf-4adb-9c93-67bdc19e0a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.473024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d0d51-12cf-4adb-9c93-67bdc19e0a23 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.476238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d0d51-12cf-4adb-9c93-67bdc19e0a23 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bd04aa0eebc4c8390eec35a93467757, group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:12.477720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d0d51-12cf-4adb-9c93-67bdc19e0a23 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:12.501289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fda40-4039-4b81-ad6f-5655f48c25cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:12.502095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fda40-4039-4b81-ad6f-5655f48c25cc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.502505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fda40-4039-4b81-ad6f-5655f48c25cc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.502842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fda40-4039-4b81-ad6f-5655f48c25cc None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.503695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fda40-4039-4b81-ad6f-5655f48c25cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:12.504052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fda40-4039-4b81-ad6f-5655f48c25cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:12.505445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fda40-4039-4b81-ad6f-5655f48c25cc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:12.517907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 195/404] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:12.517907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aee453-601b-4096-bfcc-ca2861072fd8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.517907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aee453-601b-4096-bfcc-ca2861072fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.517907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aee453-601b-4096-bfcc-ca2861072fd8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.517907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 196/405] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:12.520912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aee453-601b-4096-bfcc-ca2861072fd8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:12.603940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 210/406] 10.222.0.25 () {58 vars in 1442 bytes} [Fri May 13 13:04:12 2022] PUT /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:04:12.605384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c363e-2ae2-4880-8aa4-471431d2caf5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.605708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c363e-2ae2-4880-8aa4-471431d2caf5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.605996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c363e-2ae2-4880-8aa4-471431d2caf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.646751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-583c363e-2ae2-4880-8aa4-471431d2caf5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:12.656882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-583c363e-2ae2-4880-8aa4-471431d2caf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:12.676150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aee453-601b-4096-bfcc-ca2861072fd8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:12.737403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0a229-0810-47e6-8f98-79a476896149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:12.738761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a229-0810-47e6-8f98-79a476896149 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.739252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a229-0810-47e6-8f98-79a476896149 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.739796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a229-0810-47e6-8f98-79a476896149 None admin] PATH_INFO: `/v3/groups/4ff4dc1e9557499db5d4c29b994a0cc5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.741319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0a229-0810-47e6-8f98-79a476896149 None admin] RBAC: Authorizing `identity:get_group(group_id=4ff4dc1e9557499db5d4c29b994a0cc5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:12.743522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0a229-0810-47e6-8f98-79a476896149 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:12.752449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 197/407] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:12 2022] GET /identity/v3/groups/4ff4dc1e9557499db5d4c29b994a0cc5 => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:12.757912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0a229-0810-47e6-8f98-79a476896149 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:12.774465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 211/408] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:12.816077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c363e-2ae2-4880-8aa4-471431d2caf5 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:12.883811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0a229-0810-47e6-8f98-79a476896149 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:12.918815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6723f16-0b4e-471e-8deb-cd836b5ae712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:12.919701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6723f16-0b4e-471e-8deb-cd836b5ae712 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:12.919992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6723f16-0b4e-471e-8deb-cd836b5ae712 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:12.920272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6723f16-0b4e-471e-8deb-cd836b5ae712 None admin] PATH_INFO: `/v3/projects/989de7565a074e299634e689ed15d9f9` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:12.921522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6723f16-0b4e-471e-8deb-cd836b5ae712 None admin] RBAC: Authorizing `identity:get_project(project_id=989de7565a074e299634e689ed15d9f9)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:12.923109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6723f16-0b4e-471e-8deb-cd836b5ae712 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:12.924760 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c363e-2ae2-4880-8aa4-471431d2caf5 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:12.925361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 198/409] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:12 2022] GET /identity/v3/projects/989de7565a074e299634e689ed15d9f9 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:12.931539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6723f16-0b4e-471e-8deb-cd836b5ae712 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:13.001055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722ede7-6dbe-4481-951f-9b7f0cf2d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:13.001934 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722ede7-6dbe-4481-951f-9b7f0cf2d466 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.002214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722ede7-6dbe-4481-951f-9b7f0cf2d466 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.005082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722ede7-6dbe-4481-951f-9b7f0cf2d466 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.023000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722ede7-6dbe-4481-951f-9b7f0cf2d466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-7722ede7-6dbe-4481-951f-9b7f0cf2d466 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:13.035173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:13.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:13.036871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:13.043054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 212/410] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:04:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:13.050453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722ede7-6dbe-4481-951f-9b7f0cf2d466 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:13.080066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6723f16-0b4e-471e-8deb-cd836b5ae712 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:13.121368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b70a6-c247-4eb4-823d-f0dec0748786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:13.122287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b70a6-c247-4eb4-823d-f0dec0748786 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.122572 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b70a6-c247-4eb4-823d-f0dec0748786 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.122850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b70a6-c247-4eb4-823d-f0dec0748786 None admin] PATH_INFO: `/v3/projects/989de7565a074e299634e689ed15d9f9/groups/4ff4dc1e9557499db5d4c29b994a0cc5/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.124856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b70a6-c247-4eb4-823d-f0dec0748786 None admin] RBAC: Authorizing `identity:create_grant(project_id=989de7565a074e299634e689ed15d9f9, group_id=4ff4dc1e9557499db5d4c29b994a0cc5, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:13.126342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b70a6-c247-4eb4-823d-f0dec0748786 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:13.160277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722ede7-6dbe-4481-951f-9b7f0cf2d466 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:13.194663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f9a5f-4069-42b3-8894-0867c8fef7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:13.195326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f9a5f-4069-42b3-8894-0867c8fef7bd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.195608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f9a5f-4069-42b3-8894-0867c8fef7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.195880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f9a5f-4069-42b3-8894-0867c8fef7bd None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.196611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f9a5f-4069-42b3-8894-0867c8fef7bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:13.196916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f9a5f-4069-42b3-8894-0867c8fef7bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:13.198404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f9a5f-4069-42b3-8894-0867c8fef7bd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:13.206803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 213/411] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:13.209851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834e47-41b5-4933-8a8c-6a64f91d54a9 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.209851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834e47-41b5-4933-8a8c-6a64f91d54a9 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.209851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f834e47-41b5-4933-8a8c-6a64f91d54a9 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.210074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 214/412] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:13.212798 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f834e47-41b5-4933-8a8c-6a64f91d54a9 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:13.238992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 199/413] 10.222.0.25 () {58 vars in 1442 bytes} [Fri May 13 13:04:12 2022] PUT /identity/v3/projects/989de7565a074e299634e689ed15d9f9/groups/4ff4dc1e9557499db5d4c29b994a0cc5/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:04:13.240641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7ab7-f9e2-4f1a-a682-7861ab27f3d9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.241272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7ab7-f9e2-4f1a-a682-7861ab27f3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.241820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c7ab7-f9e2-4f1a-a682-7861ab27f3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.267110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-493c7ab7-f9e2-4f1a-a682-7861ab27f3d9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:13.268738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-493c7ab7-f9e2-4f1a-a682-7861ab27f3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:13.393732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f834e47-41b5-4933-8a8c-6a64f91d54a9 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:13.407490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 200/414] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:13.432010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c7ab7-f9e2-4f1a-a682-7861ab27f3d9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:13.453214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95655a2-1779-4ed7-938a-bd273024f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:13.454126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95655a2-1779-4ed7-938a-bd273024f514 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.454407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95655a2-1779-4ed7-938a-bd273024f514 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.454679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95655a2-1779-4ed7-938a-bd273024f514 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.456632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95655a2-1779-4ed7-938a-bd273024f514 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:13.458121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95655a2-1779-4ed7-938a-bd273024f514 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:13.460028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 215/415] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:13 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:13.465613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95655a2-1779-4ed7-938a-bd273024f514 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:13.549866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c7ab7-f9e2-4f1a-a682-7861ab27f3d9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:13.572515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95655a2-1779-4ed7-938a-bd273024f514 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:13.584879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24961f-2bf0-477b-8968-e4bbd575b4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:13.586063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24961f-2bf0-477b-8968-e4bbd575b4ee None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.586539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24961f-2bf0-477b-8968-e4bbd575b4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.587004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24961f-2bf0-477b-8968-e4bbd575b4ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.610523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24961f-2bf0-477b-8968-e4bbd575b4ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:13.616835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c8b31-0773-4715-8781-679c6b1efa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:13.617710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c8b31-0773-4715-8781-679c6b1efa7c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.618023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c8b31-0773-4715-8781-679c6b1efa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.618302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c8b31-0773-4715-8781-679c6b1efa7c None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6e24961f-2bf0-477b-8968-e4bbd575b4ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:13.620738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:13.621603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:13.622437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:13.623059 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c8b31-0773-4715-8781-679c6b1efa7c None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:13.623943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c8b31-0773-4715-8781-679c6b1efa7c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:13.625420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 201/416] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:04:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:13.629648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 216/417] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:13 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:13.631094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24961f-2bf0-477b-8968-e4bbd575b4ee None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:13.643699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c8b31-0773-4715-8781-679c6b1efa7c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:13.740199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24961f-2bf0-477b-8968-e4bbd575b4ee None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:13.774101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:13.775112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.775637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.776135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.777073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:13.777570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:13.779281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:13.788069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 202/418] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:13.793614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:13.821567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c8b31-0773-4715-8781-679c6b1efa7c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:13.894541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3598b-03a8-45ac-ae57-74573fe0b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:13.895820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3598b-03a8-45ac-ae57-74573fe0b21f None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:13.898410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3598b-03a8-45ac-ae57-74573fe0b21f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:13.898897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3598b-03a8-45ac-ae57-74573fe0b21f None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920/roles/9065c36fc305418baa5b977858b445dd` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:13.903052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3598b-03a8-45ac-ae57-74573fe0b21f None admin] RBAC: Authorizing `identity:create_grant(project_id=3bd04aa0eebc4c8390eec35a93467757, group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role_id=9065c36fc305418baa5b977858b445dd)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:13.906139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3598b-03a8-45ac-ae57-74573fe0b21f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:13.960100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ebc9a-0182-4b48-83e4-0b715bfc3ebc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:13.977819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 217/419] 10.222.0.25 () {58 vars in 1442 bytes} [Fri May 13 13:04:13 2022] PUT /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920/roles/9065c36fc305418baa5b977858b445dd => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:04:14.016988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07beb10b-dc73-4d72-8d23-b565741b9fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:14.018120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07beb10b-dc73-4d72-8d23-b565741b9fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:14.018602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07beb10b-dc73-4d72-8d23-b565741b9fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:14.019152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07beb10b-dc73-4d72-8d23-b565741b9fd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:14.105026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07beb10b-dc73-4d72-8d23-b565741b9fd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:14.106994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07beb10b-dc73-4d72-8d23-b565741b9fd6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:14.210978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 203/420] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:04:14.216673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3598b-03a8-45ac-ae57-74573fe0b21f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:14.332693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3598b-03a8-45ac-ae57-74573fe0b21f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:14.367248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f93ce1-f353-45f3-bc39-5cd57ccb58f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:14.368469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f93ce1-f353-45f3-bc39-5cd57ccb58f8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:14.368952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f93ce1-f353-45f3-bc39-5cd57ccb58f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:14.369415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f93ce1-f353-45f3-bc39-5cd57ccb58f8 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:14.370888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f93ce1-f353-45f3-bc39-5cd57ccb58f8 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:14.372513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f93ce1-f353-45f3-bc39-5cd57ccb58f8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:14.374815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 218/421] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:14 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:14.387219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07beb10b-dc73-4d72-8d23-b565741b9fd6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:14.460662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dca64-f2a5-4670-86be-60498df1c9db None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:14.461341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dca64-f2a5-4670-86be-60498df1c9db None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:14.461883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dca64-f2a5-4670-86be-60498df1c9db None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:14.463501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 219/422] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:04:14.469126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd7ab-a802-4877-b30a-66ab5ed432d3 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:14.469420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd7ab-a802-4877-b30a-66ab5ed432d3 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:14.469701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cd7ab-a802-4877-b30a-66ab5ed432d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:14.506907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07beb10b-dc73-4d72-8d23-b565741b9fd6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:14.522839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-399cd7ab-a802-4877-b30a-66ab5ed432d3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:14.524352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-399cd7ab-a802-4877-b30a-66ab5ed432d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:14.541876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15a27d-16bd-4aca-9a52-acf61d75b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:14.542728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15a27d-16bd-4aca-9a52-acf61d75b692 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:14.543886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15a27d-16bd-4aca-9a52-acf61d75b692 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:14.543886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15a27d-16bd-4aca-9a52-acf61d75b692 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:14.544494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15a27d-16bd-4aca-9a52-acf61d75b692 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:14.545952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15a27d-16bd-4aca-9a52-acf61d75b692 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:14.547905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 204/423] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:14 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:14.553140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15a27d-16bd-4aca-9a52-acf61d75b692 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:14.660505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15a27d-16bd-4aca-9a52-acf61d75b692 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:14.691439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 220/424] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:14.697171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:14.698016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:14.698895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:14.699263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:14.700249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] RBAC: Adding query filter params (group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=1eafe2a57502441faa06c7ab6de43b01) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:14.700655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:14.702257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:14.708914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 205/425] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:14 2022] GET /identity/v3/role_assignments?group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=9065c36fc305418baa5b977858b445dd => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:14.723139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cd7ab-a802-4877-b30a-66ab5ed432d3 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:14.831287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cd7ab-a802-4877-b30a-66ab5ed432d3 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:14.867755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ce597-1b36-4961-9341-4fce49cd9825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:14.868600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce597-1b36-4961-9341-4fce49cd9825 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:14.868879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce597-1b36-4961-9341-4fce49cd9825 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:14.869151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce597-1b36-4961-9341-4fce49cd9825 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:14.882280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ce597-1b36-4961-9341-4fce49cd9825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-0f6ce597-1b36-4961-9341-4fce49cd9825 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:14.890454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:14.892611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:14.894604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:14.897818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 221/426] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:14.901812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:15.006738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616b552-18c4-4144-84a6-cfb5d3e730a8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:15.046076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c5bef-d9ef-43b6-88c6-c7f5d87131a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:15.046762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c5bef-d9ef-43b6-88c6-c7f5d87131a0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.047047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c5bef-d9ef-43b6-88c6-c7f5d87131a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.047328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c5bef-d9ef-43b6-88c6-c7f5d87131a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.048090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c5bef-d9ef-43b6-88c6-c7f5d87131a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:15.048399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c5bef-d9ef-43b6-88c6-c7f5d87131a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:15.049804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c5bef-d9ef-43b6-88c6-c7f5d87131a0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:15.059597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 206/427] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:15.064811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ce597-1b36-4961-9341-4fce49cd9825 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:15.174542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ce597-1b36-4961-9341-4fce49cd9825 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:15.209937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d006522-5a6e-4813-a966-cc2a42e2b5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:15.210596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d006522-5a6e-4813-a966-cc2a42e2b5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.210882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d006522-5a6e-4813-a966-cc2a42e2b5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.211150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d006522-5a6e-4813-a966-cc2a42e2b5f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.279159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d006522-5a6e-4813-a966-cc2a42e2b5f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:15.280545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d006522-5a6e-4813-a966-cc2a42e2b5f8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:15.305912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ef0d7-7f0e-4134-a556-34bde4fcf4aa None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.306532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ef0d7-7f0e-4134-a556-34bde4fcf4aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.306532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ef0d7-7f0e-4134-a556-34bde4fcf4aa None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.308364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 207/428] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:15.313385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3653a-d556-4615-93f5-ab71caed5ce9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.313677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3653a-d556-4615-93f5-ab71caed5ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.313995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3653a-d556-4615-93f5-ab71caed5ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.331815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-15a3653a-d556-4615-93f5-ab71caed5ce9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:15.332999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-15a3653a-d556-4615-93f5-ab71caed5ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:15.378938 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 222/429] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:15.386890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d006522-5a6e-4813-a966-cc2a42e2b5f8 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:15.402137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 208/430] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:15.424917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3653a-d556-4615-93f5-ab71caed5ce9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:15.505836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d006522-5a6e-4813-a966-cc2a42e2b5f8 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:15.541842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3653a-d556-4615-93f5-ab71caed5ce9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:15.545048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484400d-fbbc-493b-be80-46d3a427c980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:15.545978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484400d-fbbc-493b-be80-46d3a427c980 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.546267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484400d-fbbc-493b-be80-46d3a427c980 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.546544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484400d-fbbc-493b-be80-46d3a427c980 None admin] PATH_INFO: `/v3/projects/1eafe2a57502441faa06c7ab6de43b01` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.552031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484400d-fbbc-493b-be80-46d3a427c980 None admin] RBAC: Authorizing `identity:get_project(project_id=1eafe2a57502441faa06c7ab6de43b01)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:15.555271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484400d-fbbc-493b-be80-46d3a427c980 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:15.558194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 223/431] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:15 2022] GET /identity/v3/projects/1eafe2a57502441faa06c7ab6de43b01 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:15.563292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484400d-fbbc-493b-be80-46d3a427c980 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:15.588411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bf7ba-06bc-4182-9649-3a9355afcaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:15.589472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bf7ba-06bc-4182-9649-3a9355afcaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.589811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bf7ba-06bc-4182-9649-3a9355afcaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.590256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649bf7ba-06bc-4182-9649-3a9355afcaf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.601497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649bf7ba-06bc-4182-9649-3a9355afcaf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-649bf7ba-06bc-4182-9649-3a9355afcaf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:15.609541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:15.610478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 13 13:04:15.611286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:15.615657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 209/432] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:15.616914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bf7ba-06bc-4182-9649-3a9355afcaf7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:15.692171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484400d-fbbc-493b-be80-46d3a427c980 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:15.721205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649bf7ba-06bc-4182-9649-3a9355afcaf7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:15.738728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d92fe-d5c1-4729-9aef-0787a6253c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:15.739893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d92fe-d5c1-4729-9aef-0787a6253c69 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.740349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d92fe-d5c1-4729-9aef-0787a6253c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.740793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d92fe-d5c1-4729-9aef-0787a6253c69 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.742723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d92fe-d5c1-4729-9aef-0787a6253c69 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:15.744643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d92fe-d5c1-4729-9aef-0787a6253c69 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:15.747933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 224/433] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:15 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:15.754078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d92fe-d5c1-4729-9aef-0787a6253c69 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:15.762631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:15.763366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.763655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.763937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.764845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] RBAC: Adding query filter params (name=member) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:15.765162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:15.766754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:15.775630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 210/434] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:04:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:15.781213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:15.873972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d92fe-d5c1-4729-9aef-0787a6253c69 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:15.907324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a3c80-5ba0-40ab-be91-ecb6c243f81e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:15.915471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af122602-76e5-43b5-b21b-13b95ca467be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:15.916134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af122602-76e5-43b5-b21b-13b95ca467be None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.916412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af122602-76e5-43b5-b21b-13b95ca467be None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.917228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af122602-76e5-43b5-b21b-13b95ca467be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:15.918392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af122602-76e5-43b5-b21b-13b95ca467be None admin] RBAC: Adding query filter params (group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=1eafe2a57502441faa06c7ab6de43b01) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:15.918714 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af122602-76e5-43b5-b21b-13b95ca467be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:15.920203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af122602-76e5-43b5-b21b-13b95ca467be None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:15.932178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 225/435] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:15 2022] GET /identity/v3/role_assignments?group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920&scope.project.id=1eafe2a57502441faa06c7ab6de43b01&role.id=3552540dc0054d2e87fad160a358c58b => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:15.950855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c5204-0566-4c7a-881f-6c1fc44c9dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:15.951907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5204-0566-4c7a-881f-6c1fc44c9dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:15.952388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5204-0566-4c7a-881f-6c1fc44c9dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:15.952883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5204-0566-4c7a-881f-6c1fc44c9dd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.027299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c5204-0566-4c7a-881f-6c1fc44c9dd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:16.029163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c5204-0566-4c7a-881f-6c1fc44c9dd1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:16.062517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d855b-d002-4bda-bb83-1e2353ab98d4 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.062517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d855b-d002-4bda-bb83-1e2353ab98d4 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.062802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d855b-d002-4bda-bb83-1e2353ab98d4 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.064523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 226/436] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:04:16.070665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3356-8239-4cbe-aed1-6d6f5c0ced42 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.071035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3356-8239-4cbe-aed1-6d6f5c0ced42 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.071318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3356-8239-4cbe-aed1-6d6f5c0ced42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.089611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-4d3e3356-8239-4cbe-aed1-6d6f5c0ced42 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:16.090866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4d3e3356-8239-4cbe-aed1-6d6f5c0ced42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:16.137594 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 211/437] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:04:16.143074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c5204-0566-4c7a-881f-6c1fc44c9dd1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:16.168772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 227/438] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:16.192177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e3356-8239-4cbe-aed1-6d6f5c0ced42 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:16.259951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c5204-0566-4c7a-881f-6c1fc44c9dd1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:16.303590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e3356-8239-4cbe-aed1-6d6f5c0ced42 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:16.305059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757add2-2621-4178-8141-14aadb858165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:16.305961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757add2-2621-4178-8141-14aadb858165 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.306504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757add2-2621-4178-8141-14aadb858165 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.306504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757add2-2621-4178-8141-14aadb858165 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.308504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757add2-2621-4178-8141-14aadb858165 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:16.310030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757add2-2621-4178-8141-14aadb858165 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:16.311763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 212/439] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:16.317047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757add2-2621-4178-8141-14aadb858165 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:16.350437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38596e-2b67-476a-a375-798d429874ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:16.351265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38596e-2b67-476a-a375-798d429874ce None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.351545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38596e-2b67-476a-a375-798d429874ce None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.351815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38596e-2b67-476a-a375-798d429874ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.364349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38596e-2b67-476a-a375-798d429874ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-0c38596e-2b67-476a-a375-798d429874ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:16.372140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:16.373063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:16.374184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 228/440] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:16.379353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38596e-2b67-476a-a375-798d429874ce None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:16.439683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757add2-2621-4178-8141-14aadb858165 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:16.483975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38240-5636-4565-afac-f70111258641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:16.485236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38240-5636-4565-afac-f70111258641 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.485724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38240-5636-4565-afac-f70111258641 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.486332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38240-5636-4565-afac-f70111258641 None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.488664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc38240-5636-4565-afac-f70111258641 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:16.490478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc38240-5636-4565-afac-f70111258641 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:16.492496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 213/441] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:16.500609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38240-5636-4565-afac-f70111258641 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:16.506992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38596e-2b67-476a-a375-798d429874ce None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:16.554604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:16.555280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.555567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.555836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.556592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:16.556902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:16.558650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:16.567457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 229/442] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:16.574277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:16.640083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38240-5636-4565-afac-f70111258641 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:16.680259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c13e4a-b4d1-4e4c-9e62-2b88c37f4272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:16.680908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13e4a-b4d1-4e4c-9e62-2b88c37f4272 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.681187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13e4a-b4d1-4e4c-9e62-2b88c37f4272 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.681455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13e4a-b4d1-4e4c-9e62-2b88c37f4272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.682383 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c13e4a-b4d1-4e4c-9e62-2b88c37f4272 None admin] RBAC: Adding query filter params (group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role.id=3552540dc0054d2e87fad160a358c58b, scope.project.id=3bd04aa0eebc4c8390eec35a93467757) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:16.682694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c13e4a-b4d1-4e4c-9e62-2b88c37f4272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:16.684386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c13e4a-b4d1-4e4c-9e62-2b88c37f4272 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:16.690142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 214/443] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/role_assignments?group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=3552540dc0054d2e87fad160a358c58b => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:16.691390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a255b4-83df-4341-9c54-0f1a4c25f83e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.691744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a255b4-83df-4341-9c54-0f1a4c25f83e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.692087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a255b4-83df-4341-9c54-0f1a4c25f83e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.692946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf12cbc-00fd-4a12-93c1-60e63e7b58ab None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:16.693632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 215/444] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:16.699716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41682a-4f8a-4d93-b358-e2d6f0e257ae None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.700049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41682a-4f8a-4d93-b358-e2d6f0e257ae None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.700272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41682a-4f8a-4d93-b358-e2d6f0e257ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.717776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3e41682a-4f8a-4d93-b358-e2d6f0e257ae None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:16.719033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3e41682a-4f8a-4d93-b358-e2d6f0e257ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:16.730850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce22591-0175-47db-adb6-ae75393e4d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:16.731928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce22591-0175-47db-adb6-ae75393e4d57 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.732466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce22591-0175-47db-adb6-ae75393e4d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.732939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce22591-0175-47db-adb6-ae75393e4d57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.786811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 216/445] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:16.805450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce22591-0175-47db-adb6-ae75393e4d57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:16.808213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce22591-0175-47db-adb6-ae75393e4d57 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:16.811003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41682a-4f8a-4d93-b358-e2d6f0e257ae None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:16.915479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 230/446] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:16.919513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41682a-4f8a-4d93-b358-e2d6f0e257ae None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:16.921649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce22591-0175-47db-adb6-ae75393e4d57 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:16.961956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77526e-4cab-480f-b36b-547dd405b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:16.962763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77526e-4cab-480f-b36b-547dd405b94c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:16.963055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77526e-4cab-480f-b36b-547dd405b94c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:16.963329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77526e-4cab-480f-b36b-547dd405b94c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:16.974608 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77526e-4cab-480f-b36b-547dd405b94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1e77526e-4cab-480f-b36b-547dd405b94c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:16.982701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:16.983593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:16.984662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 217/447] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:16.989769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77526e-4cab-480f-b36b-547dd405b94c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:17.030297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce22591-0175-47db-adb6-ae75393e4d57 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:17.064402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3573430-7a16-4d82-9f46-6b654abfbce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:17.064835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3573430-7a16-4d82-9f46-6b654abfbce2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:17.065117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3573430-7a16-4d82-9f46-6b654abfbce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:17.065396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3573430-7a16-4d82-9f46-6b654abfbce2 None admin] PATH_INFO: `/v3/projects/989de7565a074e299634e689ed15d9f9` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:17.066693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3573430-7a16-4d82-9f46-6b654abfbce2 None admin] RBAC: Authorizing `identity:get_project(project_id=989de7565a074e299634e689ed15d9f9)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:17.068149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3573430-7a16-4d82-9f46-6b654abfbce2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:17.070029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 231/448] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/projects/989de7565a074e299634e689ed15d9f9 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:17.075430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3573430-7a16-4d82-9f46-6b654abfbce2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:17.101602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77526e-4cab-480f-b36b-547dd405b94c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:17.133359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:17.134053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:17.134369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:17.134647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:17.135433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:17.135746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:17.137133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:17.149077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 218/449] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:17.154321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:17.173598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3573430-7a16-4d82-9f46-6b654abfbce2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:17.208882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99619cc-9dfe-4590-bddb-49d915e77754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:17.209738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99619cc-9dfe-4590-bddb-49d915e77754 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:17.210051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99619cc-9dfe-4590-bddb-49d915e77754 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:17.210327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99619cc-9dfe-4590-bddb-49d915e77754 None admin] PATH_INFO: `/v3/groups/4ff4dc1e9557499db5d4c29b994a0cc5` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:17.211518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99619cc-9dfe-4590-bddb-49d915e77754 None admin] RBAC: Authorizing `identity:get_group(group_id=4ff4dc1e9557499db5d4c29b994a0cc5)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:17.213007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99619cc-9dfe-4590-bddb-49d915e77754 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:17.217679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 232/450] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:17 2022] GET /identity/v3/groups/4ff4dc1e9557499db5d4c29b994a0cc5 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:17.223350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99619cc-9dfe-4590-bddb-49d915e77754 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:17.249441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b570-713a-40be-83d9-a48e40c57232 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:17.280530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952c0b1-8407-4213-9ae6-76e562c69059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:17.281166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952c0b1-8407-4213-9ae6-76e562c69059 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:17.281446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952c0b1-8407-4213-9ae6-76e562c69059 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:17.281716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952c0b1-8407-4213-9ae6-76e562c69059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:17.316428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99619cc-9dfe-4590-bddb-49d915e77754 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:17.347151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952c0b1-8407-4213-9ae6-76e562c69059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:17.348368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:17.348595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952c0b1-8407-4213-9ae6-76e562c69059 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:17.349016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:17.349297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:17.349566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:17.350420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] RBAC: Adding query filter params (group.id=4ff4dc1e9557499db5d4c29b994a0cc5, role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=989de7565a074e299634e689ed15d9f9) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:17.350723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:17.352129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:17.357704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 233/451] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:17 2022] GET /identity/v3/role_assignments?group.id=4ff4dc1e9557499db5d4c29b994a0cc5&scope.project.id=989de7565a074e299634e689ed15d9f9&role.id=663341777b0b44128ed78fe8d40f6462 => generated 611 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:17.448421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 219/452] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:04:17.454131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:17.553135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94dff46-167e-4e91-a51f-1851f44622f8 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:17.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc64b93-b4f6-41de-9726-21f5dc3d97a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:17.587913 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc64b93-b4f6-41de-9726-21f5dc3d97a1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:17.588192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc64b93-b4f6-41de-9726-21f5dc3d97a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:17.588460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc64b93-b4f6-41de-9726-21f5dc3d97a1 None admin] PATH_INFO: `/v3/projects/3bd04aa0eebc4c8390eec35a93467757` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:17.590221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc64b93-b4f6-41de-9726-21f5dc3d97a1 None admin] RBAC: Authorizing `identity:get_project(project_id=3bd04aa0eebc4c8390eec35a93467757)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:17.591656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc64b93-b4f6-41de-9726-21f5dc3d97a1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:17.593327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 234/453] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:04:17 2022] GET /identity/v3/projects/3bd04aa0eebc4c8390eec35a93467757 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:17.598946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952c0b1-8407-4213-9ae6-76e562c69059 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:17.692631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952c0b1-8407-4213-9ae6-76e562c69059 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:17.724205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713177-2541-41c0-9da8-8802b937645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:17.725045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713177-2541-41c0-9da8-8802b937645b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:17.725329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713177-2541-41c0-9da8-8802b937645b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:17.725606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713177-2541-41c0-9da8-8802b937645b None admin] PATH_INFO: `/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:17.726795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b713177-2541-41c0-9da8-8802b937645b None admin] RBAC: Authorizing `identity:get_group(group_id=1cc9aaa9ab6c4fb7a13a511dcc2c8920)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:17.728267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b713177-2541-41c0-9da8-8802b937645b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:17.730049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 220/454] 10.222.0.25 () {56 vars in 1180 bytes} [Fri May 13 13:04:17 2022] GET /identity/v3/groups/1cc9aaa9ab6c4fb7a13a511dcc2c8920 => generated 226 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:17.736822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc64b93-b4f6-41de-9726-21f5dc3d97a1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:17.831528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc64b93-b4f6-41de-9726-21f5dc3d97a1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:17.862129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88470e69-1a54-4c9e-b282-8e0606a14aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:17.862773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470e69-1a54-4c9e-b282-8e0606a14aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:17.863055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470e69-1a54-4c9e-b282-8e0606a14aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:17.863328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470e69-1a54-4c9e-b282-8e0606a14aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:17.864144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88470e69-1a54-4c9e-b282-8e0606a14aa2 None admin] RBAC: Adding query filter params (group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920, role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=3bd04aa0eebc4c8390eec35a93467757) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:17.864454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88470e69-1a54-4c9e-b282-8e0606a14aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:17.865906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88470e69-1a54-4c9e-b282-8e0606a14aa2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:17.871912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 235/455] 10.222.0.25 () {56 vars in 1376 bytes} [Fri May 13 13:04:17 2022] GET /identity/v3/role_assignments?group.id=1cc9aaa9ab6c4fb7a13a511dcc2c8920&scope.project.id=3bd04aa0eebc4c8390eec35a93467757&role.id=9065c36fc305418baa5b977858b445dd => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:20.640231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5ecf4-59a6-494a-8f82-76d8bcc6e6bc None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:20.641258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5ecf4-59a6-494a-8f82-76d8bcc6e6bc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:20.641745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5ecf4-59a6-494a-8f82-76d8bcc6e6bc None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:20.643598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 221/456] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:20.649192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa84a-76be-4fe2-85eb-ec92160f1c1e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:20.650068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa84a-76be-4fe2-85eb-ec92160f1c1e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:20.650554 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa84a-76be-4fe2-85eb-ec92160f1c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:20.674787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-13faa84a-76be-4fe2-85eb-ec92160f1c1e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:20.676361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-13faa84a-76be-4fe2-85eb-ec92160f1c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:20.771480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 236/457] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:20.796118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5ecf4-59a6-494a-8f82-76d8bcc6e6bc None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:20.917699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5ecf4-59a6-494a-8f82-76d8bcc6e6bc None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:20.959472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06ed36-ed3d-43a6-9552-33da96006153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:20.960544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06ed36-ed3d-43a6-9552-33da96006153 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:20.960974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06ed36-ed3d-43a6-9552-33da96006153 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:20.961391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06ed36-ed3d-43a6-9552-33da96006153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:20.972370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06ed36-ed3d-43a6-9552-33da96006153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:20.974102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06ed36-ed3d-43a6-9552-33da96006153 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ed06ed36-ed3d-43a6-9552-33da96006153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:20.992748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:20.993865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:20.995423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:20.995423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:20.995423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:20.995423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:21.006360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 222/458] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:21.008999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faa84a-76be-4fe2-85eb-ec92160f1c1e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:21.123589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb950d5-fb25-493b-afc9-5ac07674f1ee None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.124194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb950d5-fb25-493b-afc9-5ac07674f1ee None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.124701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb950d5-fb25-493b-afc9-5ac07674f1ee None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.126457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 223/459] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:04:21.131740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40335-ca73-4b63-8c3a-f4b9479b2bfa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.133415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40335-ca73-4b63-8c3a-f4b9479b2bfa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.133949 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d40335-ca73-4b63-8c3a-f4b9479b2bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.161168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-48d40335-ca73-4b63-8c3a-f4b9479b2bfa None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:21.162813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-48d40335-ca73-4b63-8c3a-f4b9479b2bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:21.177689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faa84a-76be-4fe2-85eb-ec92160f1c1e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:21.220795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a3fa-8110-4350-83e1-510d3eca8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:21.222119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a3fa-8110-4350-83e1-510d3eca8e7b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.222589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a3fa-8110-4350-83e1-510d3eca8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.223239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a3fa-8110-4350-83e1-510d3eca8e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.224479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481a3fa-8110-4350-83e1-510d3eca8e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:21.224988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481a3fa-8110-4350-83e1-510d3eca8e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:21.227083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481a3fa-8110-4350-83e1-510d3eca8e7b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:21.244819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 237/460] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:21.246521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb8700-deb7-4aba-b583-390e194ec4fc None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.246993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb8700-deb7-4aba-b583-390e194ec4fc None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.247450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb8700-deb7-4aba-b583-390e194ec4fc None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.249652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 238/461] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:21.251114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb8700-deb7-4aba-b583-390e194ec4fc None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:21.270764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 224/462] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:21.272167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fde4c-b270-4d00-82a6-e321b96d5ee9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.272626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fde4c-b270-4d00-82a6-e321b96d5ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.273083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fde4c-b270-4d00-82a6-e321b96d5ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.303905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b14fde4c-b270-4d00-82a6-e321b96d5ee9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:21.306202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b14fde4c-b270-4d00-82a6-e321b96d5ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:21.384842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 225/463] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:21.390926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fde4c-b270-4d00-82a6-e321b96d5ee9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:21.396578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb8700-deb7-4aba-b583-390e194ec4fc None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:21.431844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972ee0e-db3b-448e-867f-a5407475a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:21.433594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ee0e-db3b-448e-867f-a5407475a4ea None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.434078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ee0e-db3b-448e-867f-a5407475a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.434528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ee0e-db3b-448e-867f-a5407475a4ea None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.435431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972ee0e-db3b-448e-867f-a5407475a4ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:21.437038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972ee0e-db3b-448e-867f-a5407475a4ea None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:21.523652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fde4c-b270-4d00-82a6-e321b96d5ee9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:21.554612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 239/464] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:04:21 2022] POST /identity/v3/users => generated 246 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:21.555771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972ee0e-db3b-448e-867f-a5407475a4ea None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:21.574720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f603a-71d2-400d-9b56-356155b65bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:21.576008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f603a-71d2-400d-9b56-356155b65bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.577006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f603a-71d2-400d-9b56-356155b65bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.577662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f603a-71d2-400d-9b56-356155b65bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.593999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f603a-71d2-400d-9b56-356155b65bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:21.596439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f603a-71d2-400d-9b56-356155b65bd8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1e5f603a-71d2-400d-9b56-356155b65bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:21.627864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:21.628870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:21.636644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:21.636644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:21.636644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:21.636644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:21.639043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 226/465] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:21.644476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61aef-9f6e-4e7f-9448-006352fd0241 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.646151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61aef-9f6e-4e7f-9448-006352fd0241 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.646679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61aef-9f6e-4e7f-9448-006352fd0241 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.649257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 227/466] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:21.650452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61aef-9f6e-4e7f-9448-006352fd0241 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:21.717520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972ee0e-db3b-448e-867f-a5407475a4ea None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:21.764432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ecd21b-ea06-408c-beb9-6b7143f99b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:21.765632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecd21b-ea06-408c-beb9-6b7143f99b6d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.766188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecd21b-ea06-408c-beb9-6b7143f99b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.766776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ecd21b-ea06-408c-beb9-6b7143f99b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.769312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61aef-9f6e-4e7f-9448-006352fd0241 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:21.779834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecd21b-ea06-408c-beb9-6b7143f99b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:21.782361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ecd21b-ea06-408c-beb9-6b7143f99b6d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-d8ecd21b-ea06-408c-beb9-6b7143f99b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:21.812779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:21.814428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:21.815834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:21.815834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:21.815834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:21.815834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:21.819088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:21.820026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.820456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.823509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.823509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:21.823690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 240/467] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:21.823774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:21.823961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07263a43-bf3d-4b2d-a7ab-c8440163d72c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:21.824255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07263a43-bf3d-4b2d-a7ab-c8440163d72c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:21.824532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07263a43-bf3d-4b2d-a7ab-c8440163d72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:21.826381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:21.840032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 228/468] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:21.841089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:21.846522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-07263a43-bf3d-4b2d-a7ab-c8440163d72c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:21.847998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-07263a43-bf3d-4b2d-a7ab-c8440163d72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:21.930793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 241/469] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:21.931813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07263a43-bf3d-4b2d-a7ab-c8440163d72c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:21.984168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2873ac0-8bd7-4236-8e18-6881930de75e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:22.024050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:22.025081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.025368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.025643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.026950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:22.027576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:22.029270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:22.044118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 229/470] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:22.045651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:22.063903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07263a43-bf3d-4b2d-a7ab-c8440163d72c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:22.101255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0160c7-affe-4623-85af-9024e252bd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:22.102163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0160c7-affe-4623-85af-9024e252bd59 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.102570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0160c7-affe-4623-85af-9024e252bd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.102860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0160c7-affe-4623-85af-9024e252bd59 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.103782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0160c7-affe-4623-85af-9024e252bd59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:22.106121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0160c7-affe-4623-85af-9024e252bd59 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:22.160417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92aeb91-b71d-4857-8930-f9ea1a4b8f67 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:22.190337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 242/471] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:04:21 2022] POST /identity/v3/users => generated 248 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:22.191285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0160c7-affe-4623-85af-9024e252bd59 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:22.224580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:22.225399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.225680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.225981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.248487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:22.250020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:22.283879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:22.284799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:22.285639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:22.285639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:22.285639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:22.285639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:22.285639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 230/472] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:22.294886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:22.341715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0160c7-affe-4623-85af-9024e252bd59 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:22.390148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cc796-2fad-4a98-a6f0-3c4074426880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:22.390823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cc796-2fad-4a98-a6f0-3c4074426880 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.391129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cc796-2fad-4a98-a6f0-3c4074426880 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.391450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cc796-2fad-4a98-a6f0-3c4074426880 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.392753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cc796-2fad-4a98-a6f0-3c4074426880 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:22.393573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cc796-2fad-4a98-a6f0-3c4074426880 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:22.431672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45bf4e-3f46-47ef-9e8e-e2849ab70ee4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:22.465744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee88051-4cd6-42ff-97cc-aecc49fb7074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:22.466372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88051-4cd6-42ff-97cc-aecc49fb7074 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.466649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88051-4cd6-42ff-97cc-aecc49fb7074 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.466919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88051-4cd6-42ff-97cc-aecc49fb7074 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.467639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88051-4cd6-42ff-97cc-aecc49fb7074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:22.467941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88051-4cd6-42ff-97cc-aecc49fb7074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:22.469376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88051-4cd6-42ff-97cc-aecc49fb7074 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:22.483879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 231/473] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:22.483879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6469-9044-41ef-924a-de7c7935d1ed None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.484278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6469-9044-41ef-924a-de7c7935d1ed None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.484564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6469-9044-41ef-924a-de7c7935d1ed None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.489305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 232/474] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:22.490730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e6469-9044-41ef-924a-de7c7935d1ed None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:22.492175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 243/475] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:04:22 2022] POST /identity/v3/users => generated 248 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:22.499217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc7a21-8437-402b-9b9a-6e0b98f60824 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.499837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc7a21-8437-402b-9b9a-6e0b98f60824 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.500389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc7a21-8437-402b-9b9a-6e0b98f60824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.524107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-30fc7a21-8437-402b-9b9a-6e0b98f60824 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:22.526804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-30fc7a21-8437-402b-9b9a-6e0b98f60824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:22.610510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e6469-9044-41ef-924a-de7c7935d1ed None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:22.619878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 244/476] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:22.652425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc7a21-8437-402b-9b9a-6e0b98f60824 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:22.665514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15beb-66de-42d2-9335-b274e12887c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:22.666283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15beb-66de-42d2-9335-b274e12887c9 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.667167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15beb-66de-42d2-9335-b274e12887c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.667446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e15beb-66de-42d2-9335-b274e12887c9 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.668183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e15beb-66de-42d2-9335-b274e12887c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:22.669743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e15beb-66de-42d2-9335-b274e12887c9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:22.751482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 233/477] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:04:22 2022] POST /identity/v3/users => generated 249 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:22.762562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc7a21-8437-402b-9b9a-6e0b98f60824 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:22.802739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:22.803436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:22.803725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:22.804008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:22.804765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:22.806213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:22.857856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.sql.core [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 13:04:22.857856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 13 13:04:22.857856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [parameters: {'id': '73668753cc514bb287925a3c6a6ba4db', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 13 13:04:22.857856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80278) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application conn.query(q) May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application result.read() May 13 13:04:22.878282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return ref.to_dict() May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:04:22.879241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application next(self.gen) May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application yield resource May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application next(self.gen) May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self.session.rollback() May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self.force_reraise() May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise self.value May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application session.commit() May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 13:04:22.880163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._prepare_impl() May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self.session.flush() May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._flush(objects) May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application compat.raise_( May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application flush_context.execute() May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rec.execute(self) May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 13:04:22.881763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application c = connection._execute_20( May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:04:22.888378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application conn.query(q) May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application result.read() May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:04:22.890477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [parameters: {'id': '73668753cc514bb287925a3c6a6ba4db', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:22.892475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 13 13:04:22.894488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:22.897259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 245/478] 10.222.0.25 () {60 vars in 1131 bytes} [Fri May 13 13:04:22 2022] POST /identity/v3/roles => generated 144 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:22.897397 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15beb-66de-42d2-9335-b274e12887c9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:23.019529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e15beb-66de-42d2-9335-b274e12887c9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:23.054059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fac509-8dd4-452c-a87e-4eb7a8d039f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:23.055252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fac509-8dd4-452c-a87e-4eb7a8d039f5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:23.055782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fac509-8dd4-452c-a87e-4eb7a8d039f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:23.056255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fac509-8dd4-452c-a87e-4eb7a8d039f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:23.066912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fac509-8dd4-452c-a87e-4eb7a8d039f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-86fac509-8dd4-452c-a87e-4eb7a8d039f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:23.075401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:23.076312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 13 13:04:23.077160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:23.079928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 234/479] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:04:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:23.085604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:23.192419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2442c-345a-4c42-991d-cbb5ae6dc4af None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:23.227394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e26df-a1d5-4c43-a363-d25657b19a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:23.228453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e26df-a1d5-4c43-a363-d25657b19a35 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:23.228962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e26df-a1d5-4c43-a363-d25657b19a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:23.229445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e26df-a1d5-4c43-a363-d25657b19a35 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:23.230417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e26df-a1d5-4c43-a363-d25657b19a35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:23.230926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e26df-a1d5-4c43-a363-d25657b19a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:23.232517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e26df-a1d5-4c43-a363-d25657b19a35 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:23.241427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 246/480] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:24.307872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4543-ae6a-4371-af68-c3a00d94acf8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:24.308631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4543-ae6a-4371-af68-c3a00d94acf8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:24.308734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce4543-ae6a-4371-af68-c3a00d94acf8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:24.310912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 235/481] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:24.316545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830c51-e252-4505-8730-32d8833ba9e6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:24.317451 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830c51-e252-4505-8730-32d8833ba9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:24.317975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830c51-e252-4505-8730-32d8833ba9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:24.346755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-38830c51-e252-4505-8730-32d8833ba9e6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:24.347953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-38830c51-e252-4505-8730-32d8833ba9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:24.459446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 247/482] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:24.498922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce4543-ae6a-4371-af68-c3a00d94acf8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:24.634607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce4543-ae6a-4371-af68-c3a00d94acf8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:24.672768 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246ad3e-6d01-4b5e-8504-9b77052bb96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:24.673562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246ad3e-6d01-4b5e-8504-9b77052bb96b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:24.673870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246ad3e-6d01-4b5e-8504-9b77052bb96b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:24.674148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246ad3e-6d01-4b5e-8504-9b77052bb96b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:24.690704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246ad3e-6d01-4b5e-8504-9b77052bb96b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f246ad3e-6d01-4b5e-8504-9b77052bb96b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:24.698453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:24.699325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:24.700144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:24.704021 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 236/483] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:24.705311 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38830c51-e252-4505-8730-32d8833ba9e6 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:24.831020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38830c51-e252-4505-8730-32d8833ba9e6 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:24.864477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd023f2-1e68-486b-a991-a5aceb18ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:24.865149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd023f2-1e68-486b-a991-a5aceb18ff16 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:24.865430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd023f2-1e68-486b-a991-a5aceb18ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:24.865700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd023f2-1e68-486b-a991-a5aceb18ff16 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:24.866464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd023f2-1e68-486b-a991-a5aceb18ff16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:24.866767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd023f2-1e68-486b-a991-a5aceb18ff16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:24.868281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd023f2-1e68-486b-a991-a5aceb18ff16 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:24.877105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 248/484] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:24.882537 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246ad3e-6d01-4b5e-8504-9b77052bb96b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:25.025445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246ad3e-6d01-4b5e-8504-9b77052bb96b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:25.029611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7db82-4092-48dc-ad19-a9c692f7a87e None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.029930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7db82-4092-48dc-ad19-a9c692f7a87e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.030209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7db82-4092-48dc-ad19-a9c692f7a87e None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.031697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 249/485] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:04:25.037425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52f8b6-dc98-497b-baa3-45c80ca01861 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.037716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52f8b6-dc98-497b-baa3-45c80ca01861 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.038022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52f8b6-dc98-497b-baa3-45c80ca01861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.067576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-3f52f8b6-dc98-497b-baa3-45c80ca01861 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:25.069163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3f52f8b6-dc98-497b-baa3-45c80ca01861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:25.082811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:25.084000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.084484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.084972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.103577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:25.107169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:25.137660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:25.138584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:25.139449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:25.139449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:25.139449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:25.139449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:25.139449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 237/486] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:25.144851 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:25.183720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 250/487] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:25.245771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52f8b6-dc98-497b-baa3-45c80ca01861 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:25.260884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b984d-fc17-4562-b89e-e77a5fb54896 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:25.300136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:25.300786 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.301071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.301344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.302108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:25.302414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:25.303844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:25.328201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 238/488] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:25.339328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:25.444265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df63ade-34b0-462d-a07c-68094dd2ba97 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:25.491734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52f8b6-dc98-497b-baa3-45c80ca01861 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:25.493769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517578e-3690-4c62-9fce-753cfb0cd9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:25.496276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517578e-3690-4c62-9fce-753cfb0cd9ab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.496276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517578e-3690-4c62-9fce-753cfb0cd9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.496276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517578e-3690-4c62-9fce-753cfb0cd9ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.509586 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517578e-3690-4c62-9fce-753cfb0cd9ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:25.511999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517578e-3690-4c62-9fce-753cfb0cd9ab None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2517578e-3690-4c62-9fce-753cfb0cd9ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:25.545250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:25.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:25.547321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 239/489] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:04:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:25.548118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af050ebf-0946-4644-a76d-eea46b2680f3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.551365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af050ebf-0946-4644-a76d-eea46b2680f3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.551365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af050ebf-0946-4644-a76d-eea46b2680f3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.551365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 240/490] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:25.552845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af050ebf-0946-4644-a76d-eea46b2680f3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:25.555910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1bf9a9-a111-4287-af99-3abdefbdf938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:25.556914 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bf9a9-a111-4287-af99-3abdefbdf938 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.557755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bf9a9-a111-4287-af99-3abdefbdf938 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.559221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bf9a9-a111-4287-af99-3abdefbdf938 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.571846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1bf9a9-a111-4287-af99-3abdefbdf938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-dd1bf9a9-a111-4287-af99-3abdefbdf938 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:25.582095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:25.584175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 251/491] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:25.586136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3d489-2164-46bb-8ea7-ebd1837f5008 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.587378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3d489-2164-46bb-8ea7-ebd1837f5008 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.587378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3d489-2164-46bb-8ea7-ebd1837f5008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.610448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bdb3d489-2164-46bb-8ea7-ebd1837f5008 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:25.612138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bdb3d489-2164-46bb-8ea7-ebd1837f5008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:25.665330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af050ebf-0946-4644-a76d-eea46b2680f3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:25.679086 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 252/492] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:25.680019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3d489-2164-46bb-8ea7-ebd1837f5008 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:25.706785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d3eb-63f5-40ac-b416-f3812cc430b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:25.708030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d3eb-63f5-40ac-b416-f3812cc430b1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.708691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d3eb-63f5-40ac-b416-f3812cc430b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.709480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d3eb-63f5-40ac-b416-f3812cc430b1 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.710415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459d3eb-63f5-40ac-b416-f3812cc430b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:25.710735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459d3eb-63f5-40ac-b416-f3812cc430b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:25.712167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459d3eb-63f5-40ac-b416-f3812cc430b1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:25.752480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 241/493] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:04:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:25.752480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5893be4-90ab-4b51-bec1-7e637b375f2f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.752480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5893be4-90ab-4b51-bec1-7e637b375f2f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.752480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5893be4-90ab-4b51-bec1-7e637b375f2f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.752480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 242/494] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:25.752480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5893be4-90ab-4b51-bec1-7e637b375f2f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:25.793098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3d489-2164-46bb-8ea7-ebd1837f5008 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:25.829391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:25.829391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.829736 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.830119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.830965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:25.831271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:25.832673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:25.841367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 253/495] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:25.847213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:25.869855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5893be4-90ab-4b51-bec1-7e637b375f2f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:25.912187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51677b2-d5a8-4a24-9234-b4f4c9365ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:25.913297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51677b2-d5a8-4a24-9234-b4f4c9365ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.913648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51677b2-d5a8-4a24-9234-b4f4c9365ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.913955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51677b2-d5a8-4a24-9234-b4f4c9365ab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.926071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51677b2-d5a8-4a24-9234-b4f4c9365ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e51677b2-d5a8-4a24-9234-b4f4c9365ab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:25.936382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:25.937348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:25.938332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:25.943322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 243/496] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:25.944766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b559c3-b70e-4c6b-976d-016ea73b7502 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.945067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b559c3-b70e-4c6b-976d-016ea73b7502 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.945349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b559c3-b70e-4c6b-976d-016ea73b7502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:25.960689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144cf5e-1b13-4f16-b912-35e37c48bad2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:25.970207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-55b559c3-b70e-4c6b-976d-016ea73b7502 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:25.972399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-55b559c3-b70e-4c6b-976d-016ea73b7502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:25.993880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8de76-b61e-49e1-91b7-7fac0117c5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:25.995281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8de76-b61e-49e1-91b7-7fac0117c5ec None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:25.995714 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8de76-b61e-49e1-91b7-7fac0117c5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:25.996156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8de76-b61e-49e1-91b7-7fac0117c5ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.047250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 244/497] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:26.058967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b559c3-b70e-4c6b-976d-016ea73b7502 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.081997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8de76-b61e-49e1-91b7-7fac0117c5ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.083372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8de76-b61e-49e1-91b7-7fac0117c5ec None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.177644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b559c3-b70e-4c6b-976d-016ea73b7502 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.201389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 254/498] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:26.203239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8de76-b61e-49e1-91b7-7fac0117c5ec None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.216391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.217329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.217617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.217931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.227893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.229422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:26.247318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:26.248298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:26.249126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:26.249126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:26.249126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:26.249126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:26.249126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 245/499] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:26.253770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.309529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8de76-b61e-49e1-91b7-7fac0117c5ec None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.343007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.343600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.343886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.344164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.344896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:26.345206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.346611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.354459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 255/500] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:26.355123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.355343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0429e6-f2a3-4f75-8356-73a431b99a12 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.387977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4f5a6-e430-4a98-95e3-6251c261f703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.388770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4f5a6-e430-4a98-95e3-6251c261f703 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.389053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4f5a6-e430-4a98-95e3-6251c261f703 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.389331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4f5a6-e430-4a98-95e3-6251c261f703 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.399095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4f5a6-e430-4a98-95e3-6251c261f703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a1d4f5a6-e430-4a98-95e3-6251c261f703 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:26.407279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:26.408192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:26.408985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 246/501] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:26.409642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4f5a6-e430-4a98-95e3-6251c261f703 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.451084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654f8e-6c69-445a-8a16-213aefe26333 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.482616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572c80d-e9ff-4e52-aeb8-08471f77be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.483420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572c80d-e9ff-4e52-aeb8-08471f77be60 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.483705 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572c80d-e9ff-4e52-aeb8-08471f77be60 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.483983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572c80d-e9ff-4e52-aeb8-08471f77be60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.494147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572c80d-e9ff-4e52-aeb8-08471f77be60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.496197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572c80d-e9ff-4e52-aeb8-08471f77be60 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.510589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4f5a6-e430-4a98-95e3-6251c261f703 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-d572c80d-e9ff-4e52-aeb8-08471f77be60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:26.512735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:26.513628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:26.514533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:26.514533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:26.514533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:26.514533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:26.514533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 256/502] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:26.515408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba84234-6b30-4cd8-af21-360ee912f19d None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.515408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba84234-6b30-4cd8-af21-360ee912f19d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.515675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba84234-6b30-4cd8-af21-360ee912f19d None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.517041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 257/503] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:26.518886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba84234-6b30-4cd8-af21-360ee912f19d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.542390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.543043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.543323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.543595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.544305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:26.544606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.545999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.557033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 247/504] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:26.558576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.613974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba84234-6b30-4cd8-af21-360ee912f19d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.644902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.645685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.645996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.646353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.649811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694071ba-533b-43c6-a586-dc28d29bb6e3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.657345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.658828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:26.675977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:26.676954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:26.677875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:26.677875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:26.677875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:26.677875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:26.677875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 258/505] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:26.678337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.682497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f454a85-7ff3-4c00-a2ab-d440f3ec6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.683420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f454a85-7ff3-4c00-a2ab-d440f3ec6de4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.683699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f454a85-7ff3-4c00-a2ab-d440f3ec6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.683973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f454a85-7ff3-4c00-a2ab-d440f3ec6de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.684710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f454a85-7ff3-4c00-a2ab-d440f3ec6de4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:26.685011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f454a85-7ff3-4c00-a2ab-d440f3ec6de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.686401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f454a85-7ff3-4c00-a2ab-d440f3ec6de4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.694296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 248/506] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:26.695456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011087cd-0d68-4e59-bca5-d1790f9e0a48 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.695742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011087cd-0d68-4e59-bca5-d1790f9e0a48 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.696017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011087cd-0d68-4e59-bca5-d1790f9e0a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.711226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-011087cd-0d68-4e59-bca5-d1790f9e0a48 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:26.712388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-011087cd-0d68-4e59-bca5-d1790f9e0a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:26.771058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339b9f7-2956-4e01-b210-220adc8bec44 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.774029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 249/507] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:26.774646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011087cd-0d68-4e59-bca5-d1790f9e0a48 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.804130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.804793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.805074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.805347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.806088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:26.806389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.807751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.819128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 259/508] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:26.819899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.869324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011087cd-0d68-4e59-bca5-d1790f9e0a48 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.901303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.902120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.902403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.902677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.909417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.910761 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcc2e5-4e74-4994-b46b-88009d704cde None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:26.910953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:26.925985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:26.927025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:26.927941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 250/509] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:26.928526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:26.942761 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:26.943417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:26.943699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:26.943991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:26.944681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:26.944983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:26.946379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:26.957222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 260/510] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:26.958089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.027216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f6c75-c476-44b0-9c05-f90319d1f183 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.050038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542d63e-048b-4635-8946-706bec272e2e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.059668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.060450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.060736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.061009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.070549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.071994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.081705 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.082515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.082796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.083068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:27.089747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:27.090764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:27.091825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:27.091825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:27.091825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:27.091825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:27.091825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 251/511] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:27.092154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.095220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.096755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:27.113978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:27.115013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:27.115899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:27.115899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:27.115899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:27.115899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:27.115899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 261/512] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:27.116165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.194543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039d916-8b76-4ed6-b381-1729ba9f1ee5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.213372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3f931-236e-4aec-987a-8710895eaa72 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.226243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.227038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.227320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.227596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.234588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.236032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.244517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.245675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.245983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.246257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.247077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:27.247383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.248796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:27.250319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:27.251137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:27.251985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 252/513] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:27.253033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.285372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 262/514] 10.222.0.25 () {56 vars in 1137 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:27.286227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.356112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cc397-86ae-4420-81a8-6e62df8b7407 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.377071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2f99a-a9c1-478b-b9ec-42fef79f4899 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.387497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.388287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.388575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.388850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.396139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.397635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.409178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.409857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.410139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.410409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.411114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:27.411413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:27.411986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:27.412954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 13:04:27.414062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:27.414672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.414748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 253/515] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:27.414748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.423797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 263/516] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:27.424527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.511914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d0b2b-98be-4dd7-b98d-58014aca031e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.519657 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05199e4-9c11-49c9-b8ba-829517810fa7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.549294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.549974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.550256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.550528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.551239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:27.551543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.552089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.552740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.552918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.553018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.553285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.554099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:27.554408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.555812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.564441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 254/517] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:27.565209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.572842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 264/518] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:27.573646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.661197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf5ddc-a222-4d10-af4b-295b70369048 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.668170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063413b-d957-4398-9a9c-6ca4c639d112 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.693524 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bbfe6-d191-4687-a0ab-d7ca4e403303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.694280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bbfe6-d191-4687-a0ab-d7ca4e403303 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.694570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bbfe6-d191-4687-a0ab-d7ca4e403303 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.694849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bbfe6-d191-4687-a0ab-d7ca4e403303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.700686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.701354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.701639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.701953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.702784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:27.703100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.704544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.725089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 265/519] 10.222.0.25 () {56 vars in 1137 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:27.725089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.758509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bbfe6-d191-4687-a0ab-d7ca4e403303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.759901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bbfe6-d191-4687-a0ab-d7ca4e403303 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.813232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22249c33-47eb-4b6b-a7f7-5e6091532764 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.845290 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:27.846108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:27.846392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:27.846667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:27.859349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:27.860794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:27.876112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:27.876985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:27.878193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 266/520] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:27.878892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.883973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 255/521] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:27.884800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bbfe6-d191-4687-a0ab-d7ca4e403303 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:27.976914 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68795e53-44d8-44f9-b612-922fa38eb96d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:27.978333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bbfe6-d191-4687-a0ab-d7ca4e403303 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.009467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529abf9-0361-4372-8495-b8f2b147801d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.010114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529abf9-0361-4372-8495-b8f2b147801d None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.010401 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529abf9-0361-4372-8495-b8f2b147801d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.010679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529abf9-0361-4372-8495-b8f2b147801d None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.010789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.011331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.011419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529abf9-0361-4372-8495-b8f2b147801d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.011619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.011902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.012735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=3f94e52e32324d8cbf38fb05dc38ea96) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:28.012821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529abf9-0361-4372-8495-b8f2b147801d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.013050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.014512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.019797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 256/522] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:27 2022] GET /identity/v3/role_assignments?user.id=3f94e52e32324d8cbf38fb05dc38ea96&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:28.020565 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.113568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 267/523] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:04:27 2022] POST /identity/v3/users => generated 247 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:28.126890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529abf9-0361-4372-8495-b8f2b147801d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.138532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf039f-d1fb-44c0-b5a5-6a1484721c84 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.182648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0ef02-1458-437c-83d9-cba244c10564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.183438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0ef02-1458-437c-83d9-cba244c10564 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.183734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0ef02-1458-437c-83d9-cba244c10564 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.184016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0ef02-1458-437c-83d9-cba244c10564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.265188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529abf9-0361-4372-8495-b8f2b147801d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.270893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0ef02-1458-437c-83d9-cba244c10564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.272739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0ef02-1458-437c-83d9-cba244c10564 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.306259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.306929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.307253 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.307534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.308642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:28.309012 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.310421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.332297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 268/524] 10.222.0.25 () {56 vars in 1139 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:28.333536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.385938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 257/525] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:28.386895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0ef02-1458-437c-83d9-cba244c10564 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.435246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b664f6-ecf3-43c5-9fc6-8e9f1cab9f5f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.469810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.470996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.471484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.471977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.484013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.485565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.487642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0ef02-1458-437c-83d9-cba244c10564 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:28.504096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:28.505016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:28.505910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:28.505910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:28.505910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:28.505910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:28.505910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 269/526] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:28.506916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.523263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3a52-f482-4109-bc93-188d58c2b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.523847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3a52-f482-4109-bc93-188d58c2b342 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.524132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3a52-f482-4109-bc93-188d58c2b342 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.524416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3a52-f482-4109-bc93-188d58c2b342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.595063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3a52-f482-4109-bc93-188d58c2b342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.596583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3a52-f482-4109-bc93-188d58c2b342 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.610491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91983ee0-168a-4366-bc72-2c3f28a7e729 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.644651 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.645453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.645739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.646044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.656488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.657977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:28.676184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:28.678510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:28.680584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:28.680584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:28.680584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:28.680584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:28.680584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 270/527] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:28.680584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.707806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 258/528] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:28.708752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3a52-f482-4109-bc93-188d58c2b342 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.787275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da715af0-8e5c-486f-9675-6cdf4df58eca None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.812179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3a52-f482-4109-bc93-188d58c2b342 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.822207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.823391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.823675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.823951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.824673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:28.825173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.826395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.838737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 271/529] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:28.839597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.852495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.853472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.854105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.854588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.855512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:28.856041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.857685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:28.871475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 259/530] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:28.876911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:28.940057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a6c96-7978-452a-a33d-0a702ac798d4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.974622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:28.975880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:28.976189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:28.976473 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:28.979648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1330-1927-45cd-a9ed-2fb4f3d0caad None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:28.989735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:28.991875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:29.013414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:29.014510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:29.015367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:29.015367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:29.015367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:29.015367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:29.015367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 272/531] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:29.015640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.020854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.021976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.022484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.022971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.030878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.032583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:29.048436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:29.049310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:29.050269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:29.050269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:29.050269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:29.050269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:29.050269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:29.050269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:29.050269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:29.050269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:29.052384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 260/532] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:29.053377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.130323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c060f6f-99e1-4875-b3ce-0db23acc9aa8 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.165055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.166500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.166979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.167463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.167848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015a2f8-d4ad-4e58-84c2-9c07fdd7df4f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.176797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.179232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:29.201044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:29.201958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:29.202820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:29.202820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:29.202820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:29.202820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:29.202820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:29.202820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:29.202820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:29.202820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:29.208589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 273/533] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:29.208589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.215909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.216874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.217423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.218029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.218979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:29.219538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.221172 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.233492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 261/534] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:29.234676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.326345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b65e9d-ecf5-493f-a6bf-87bbd2e911c1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.343165 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca484ff-8b51-4ce1-85f9-dbb0fb332ee0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.374945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.375821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.376177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.376615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.377590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:29.378072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.379383 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.379935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.380450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.380913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.381365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.382895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:29.383396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.385043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.392548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 274/535] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:29.393394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.406046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 262/536] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:29.407927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.510221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e12228-b804-4eca-9bff-f8d64e525c09 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.541149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea6d2d-7a27-4778-8888-44f60be27f10 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.548530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.550012 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.550546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.551332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.561316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.563733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.575218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.575943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.576294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.576689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.577573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=a7955ca85d92485d91abfa401b63f26e) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:29.577972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.579504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:29.585301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:29.586171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:29.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:29.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:29.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:29.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:29.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:29.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:29.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:29.587047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:29.593744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 263/538] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/role_assignments?user.id=a7955ca85d92485d91abfa401b63f26e&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:29.594073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.594329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 275/538] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:29.595557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.714405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ba613-fab5-4ef4-9057-80bf38f6da8f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.754776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.755943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.756440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.757127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.758256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:29.758828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.762113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.776076 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 276/539] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:29.782160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:29.828909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b622426-0411-42b1-bdda-976c96825c62 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.889523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5b501-d935-4876-87ea-4634adc675ad None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:29.903086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277a516-59bc-4a35-84fa-fb92fe6e1314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.903755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277a516-59bc-4a35-84fa-fb92fe6e1314 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.904044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277a516-59bc-4a35-84fa-fb92fe6e1314 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.904327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277a516-59bc-4a35-84fa-fb92fe6e1314 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.905338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277a516-59bc-4a35-84fa-fb92fe6e1314 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=be3f786b791b4c8796d251948917ca93) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:29.905521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277a516-59bc-4a35-84fa-fb92fe6e1314 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.906961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277a516-59bc-4a35-84fa-fb92fe6e1314 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.913540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 264/540] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/role_assignments?user.id=be3f786b791b4c8796d251948917ca93&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:29.927609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b94c5-e339-4a1a-970d-a063f8141a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:29.928672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b94c5-e339-4a1a-970d-a063f8141a27 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:29.929169 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b94c5-e339-4a1a-970d-a063f8141a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:29.929672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b94c5-e339-4a1a-970d-a063f8141a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:29.930794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b94c5-e339-4a1a-970d-a063f8141a27 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=8ddb0dcaa5304016af1bc1f88845fb80) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:29.931333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b94c5-e339-4a1a-970d-a063f8141a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:29.934421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b94c5-e339-4a1a-970d-a063f8141a27 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:29.941745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 277/541] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:29 2022] GET /identity/v3/role_assignments?user.id=8ddb0dcaa5304016af1bc1f88845fb80&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:30.340245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a30f-dc5f-44ad-961e-caa69e6cdead None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:30.340649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a30f-dc5f-44ad-961e-caa69e6cdead None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:30.340827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a30f-dc5f-44ad-961e-caa69e6cdead None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:30.342482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 265/542] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:30.354989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880435-700f-4767-9a5e-33c213a653e5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:30.355290 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880435-700f-4767-9a5e-33c213a653e5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:30.355555 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83880435-700f-4767-9a5e-33c213a653e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:30.378774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-83880435-700f-4767-9a5e-33c213a653e5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:30.379987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-83880435-700f-4767-9a5e-33c213a653e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:30.460114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 278/543] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:30.487672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219192d-2699-44e1-84f9-d49ee1ad3cdd None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:30.488140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219192d-2699-44e1-84f9-d49ee1ad3cdd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:30.488213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219192d-2699-44e1-84f9-d49ee1ad3cdd None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:30.489837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 266/544] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:30.495746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cb79e-bcbf-494b-b005-a1673b617892 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:30.495977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cb79e-bcbf-494b-b005-a1673b617892 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:30.496255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cb79e-bcbf-494b-b005-a1673b617892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:30.507180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219192d-2699-44e1-84f9-d49ee1ad3cdd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:30.516959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-159cb79e-bcbf-494b-b005-a1673b617892 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:30.518187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-159cb79e-bcbf-494b-b005-a1673b617892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:30.612210 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 279/545] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:30.624707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219192d-2699-44e1-84f9-d49ee1ad3cdd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:30.657754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cb79e-bcbf-494b-b005-a1673b617892 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:30.659589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f98736-076d-465c-a1be-7d7ea34569bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:30.660403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f98736-076d-465c-a1be-7d7ea34569bc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:30.660689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f98736-076d-465c-a1be-7d7ea34569bc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:30.660966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f98736-076d-465c-a1be-7d7ea34569bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:30.672012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f98736-076d-465c-a1be-7d7ea34569bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-84f98736-076d-465c-a1be-7d7ea34569bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:30.679736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:30.680624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:30.681410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 267/546] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:30.688928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f98736-076d-465c-a1be-7d7ea34569bc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:30.766002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cb79e-bcbf-494b-b005-a1673b617892 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:30.819510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90908f-7397-4cc3-89ce-84a9e42ecc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:30.820331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90908f-7397-4cc3-89ce-84a9e42ecc59 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:30.820615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90908f-7397-4cc3-89ce-84a9e42ecc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:30.820895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90908f-7397-4cc3-89ce-84a9e42ecc59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:30.831322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f98736-076d-465c-a1be-7d7ea34569bc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:30.839094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90908f-7397-4cc3-89ce-84a9e42ecc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-7a90908f-7397-4cc3-89ce-84a9e42ecc59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:30.855526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:30.857776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:30.859759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 280/547] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:30.865870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90908f-7397-4cc3-89ce-84a9e42ecc59 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:30.866387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:30.867182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:30.867470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:30.867750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:30.868840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:30.869162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:30.871595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:30.880934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 268/548] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:30.886861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:31.009813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9c436-c898-4928-93da-8e59c9d97d39 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:31.041820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:31.041820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:31.041820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:31.041820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:31.051791 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:31.053196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:31.077810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:31.078707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:31.079546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:31.079546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:31.079546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:31.079546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:31.089917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 269/549] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:31.090771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:31.129881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90908f-7397-4cc3-89ce-84a9e42ecc59 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:31.198174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:31.198851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:31.199128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:31.201807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:31.201807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:31.201807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:31.201807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:31.231303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 281/550] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:31.243045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:31.280561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b840d72-a903-40d5-b51b-339ad2c268e4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:31.357195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:31.357893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:31.358176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:31.360602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:31.361350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:31.362589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:31.364006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:31.394267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 270/551] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:31.399551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:31.429924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaf8ef-0faa-4c8a-8d25-331caa8e97f5 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:31.466709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:31.468444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:31.468803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:31.468989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:31.487359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:31.490175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:31.523821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:31.524735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:31.526131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:31.526131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:31.526131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:31.526131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:31.532021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 282/552] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:31.539064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:31.679075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086af7cb-ab05-4b3a-9deb-c282d8dcb631 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:31.715252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:31.715922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:31.716210 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:31.716491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:31.717216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:31.717529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:31.718938 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:31.722937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b7041-4275-42fa-ba71-c29f73007ca4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:31.746004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 283/553] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:31.750009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:31.787251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:31.788059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:31.788346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:31.788620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:31.802418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:31.804071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:31.821799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:31.822806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:31.823704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 271/554] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:04:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:31.830967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:31.964991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0add1c-7be9-45ad-baa9-f1b3feb76b60 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.005195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.006025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.006417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.006703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.024796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:32.027947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:32.053711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:32.054792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:32.055684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 284/555] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:32.061521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:32.079915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdbe6f-bd29-41a9-b90a-c8d9e68e0f49 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.162663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f970eec-109e-41ca-9b06-4c1177a469dd None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.178530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.179227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.179509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.179782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.180630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:32.180936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:32.182442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:32.202293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 272/556] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:04:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:32.204976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.205667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.205985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.206268 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.207118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:32.207442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:32.208838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:32.209090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:32.228372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 285/557] 10.222.0.25 () {56 vars in 1135 bytes} [Fri May 13 13:04:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:32.233814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:32.336125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086fe2f-c4f3-4ed4-b3d6-d4683b735a58 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.375034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3e0a4-7864-4827-8cdf-59f5cd10952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.375700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e0a4-7864-4827-8cdf-59f5cd10952f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.375986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e0a4-7864-4827-8cdf-59f5cd10952f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.376265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e0a4-7864-4827-8cdf-59f5cd10952f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.423697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca87279-b2fe-4ec9-af68-fc6e3e533dc5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.445391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3e0a4-7864-4827-8cdf-59f5cd10952f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:32.446890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3e0a4-7864-4827-8cdf-59f5cd10952f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:32.526953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe7e64-21b4-4735-bed8-65f0d884296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.527692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7e64-21b4-4735-bed8-65f0d884296d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.527977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7e64-21b4-4735-bed8-65f0d884296d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.528252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7e64-21b4-4735-bed8-65f0d884296d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.546809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe7e64-21b4-4735-bed8-65f0d884296d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:32.549394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe7e64-21b4-4735-bed8-65f0d884296d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:32.559427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 286/558] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:32.562435 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa774c5-fc47-4b73-aed8-560dccc03cee None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.562861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa774c5-fc47-4b73-aed8-560dccc03cee None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.563263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa774c5-fc47-4b73-aed8-560dccc03cee None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.568846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 287/559] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:32.568846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa774c5-fc47-4b73-aed8-560dccc03cee None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a7fe7e64-21b4-4735-bed8-65f0d884296d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:32.581928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:32.582767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:32.583626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:32.583626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:32.583626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:32.583626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:32.586791 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 273/560] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:32.589065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed55e17-d2ea-4009-bde1-326b768063f1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.589499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed55e17-d2ea-4009-bde1-326b768063f1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.589941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed55e17-d2ea-4009-bde1-326b768063f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.611671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9ed55e17-d2ea-4009-bde1-326b768063f1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:32.613120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9ed55e17-d2ea-4009-bde1-326b768063f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:32.693007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa774c5-fc47-4b73-aed8-560dccc03cee None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.701840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 274/561] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:32.701840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed55e17-d2ea-4009-bde1-326b768063f1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:32.736661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb339f92-3003-4e68-872f-3952f5c96393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.737735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb339f92-3003-4e68-872f-3952f5c96393 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.738187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb339f92-3003-4e68-872f-3952f5c96393 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.738608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb339f92-3003-4e68-872f-3952f5c96393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.748513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb339f92-3003-4e68-872f-3952f5c96393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:32.750179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb339f92-3003-4e68-872f-3952f5c96393 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-eb339f92-3003-4e68-872f-3952f5c96393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:32.767511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:32.768637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:32.769694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:32.769694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:32.769694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:32.769694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:32.776134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 288/562] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:32.776134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb012-7e67-4c78-b04b-6a09fd3729b9 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.776390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb012-7e67-4c78-b04b-6a09fd3729b9 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.776520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb012-7e67-4c78-b04b-6a09fd3729b9 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.778000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 289/563] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:32.778545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdb012-7e67-4c78-b04b-6a09fd3729b9 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:32.802288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed55e17-d2ea-4009-bde1-326b768063f1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.836070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.836731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.837012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.837294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.838561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:32.838868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:32.840256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:32.851319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 275/564] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:32.852135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:32.896384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdb012-7e67-4c78-b04b-6a09fd3729b9 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.929070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84eec7d-ff1a-4bdc-9f00-472d42fbb1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.929895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eec7d-ff1a-4bdc-9f00-472d42fbb1ea None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.930184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eec7d-ff1a-4bdc-9f00-472d42fbb1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.930457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eec7d-ff1a-4bdc-9f00-472d42fbb1ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.940890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84eec7d-ff1a-4bdc-9f00-472d42fbb1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-a84eec7d-ff1a-4bdc-9f00-472d42fbb1ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:32.949170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:32.950234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 290/565] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:32.956126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eaaaf-9fba-47d9-b7be-af102324feab None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.956859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eaaaf-9fba-47d9-b7be-af102324feab None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.956859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eaaaf-9fba-47d9-b7be-af102324feab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.958008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05cf53-d55d-480c-9909-012b4266fa51 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:32.970339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-fb5eaaaf-9fba-47d9-b7be-af102324feab None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:32.971523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-fb5eaaaf-9fba-47d9-b7be-af102324feab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:32.990118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ea3ab-a37e-4cc4-973b-6042dec0285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:32.990774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea3ab-a37e-4cc4-973b-6042dec0285c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:32.991057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea3ab-a37e-4cc4-973b-6042dec0285c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:32.991333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea3ab-a37e-4cc4-973b-6042dec0285c None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:32.992083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ea3ab-a37e-4cc4-973b-6042dec0285c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:32.992386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ea3ab-a37e-4cc4-973b-6042dec0285c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:32.993749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ea3ab-a37e-4cc4-973b-6042dec0285c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.004838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 276/566] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:33.006062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295f15e-3be8-47c3-b89e-5a69470344fe None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.006350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295f15e-3be8-47c3-b89e-5a69470344fe None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.006630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295f15e-3be8-47c3-b89e-5a69470344fe None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.009943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 277/567] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:33.010765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295f15e-3be8-47c3-b89e-5a69470344fe None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.035722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 291/568] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:33.036559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eaaaf-9fba-47d9-b7be-af102324feab None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.108674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295f15e-3be8-47c3-b89e-5a69470344fe None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.131429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eaaaf-9fba-47d9-b7be-af102324feab None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.140407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.141192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.141478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.141757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.148259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.150066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.162979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18ba2a-b03a-4c6a-9ebd-7c3b907fcc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:33.163115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:33.164100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:33.165161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:33.165161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:33.165161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:33.165161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:33.165161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:33.165161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:33.165161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:33.165161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:33.165660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ba2a-b03a-4c6a-9ebd-7c3b907fcc7a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.165660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ba2a-b03a-4c6a-9ebd-7c3b907fcc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.165660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ba2a-b03a-4c6a-9ebd-7c3b907fcc7a None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.165660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18ba2a-b03a-4c6a-9ebd-7c3b907fcc7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:33.165660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18ba2a-b03a-4c6a-9ebd-7c3b907fcc7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.167094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18ba2a-b03a-4c6a-9ebd-7c3b907fcc7a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.169289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 278/569] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:33.170055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.175327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 292/570] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:33.176415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911d90-0ec6-485b-8c7a-92f34dc1f9a5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.176709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911d90-0ec6-485b-8c7a-92f34dc1f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.176993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911d90-0ec6-485b-8c7a-92f34dc1f9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.195279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2a911d90-0ec6-485b-8c7a-92f34dc1f9a5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:33.196510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2a911d90-0ec6-485b-8c7a-92f34dc1f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:33.263849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 293/571] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:33.264688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a911d90-0ec6-485b-8c7a-92f34dc1f9a5 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.266371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828390c8-a51b-409b-81e1-cb84f48f7501 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.298787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.299512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.299794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.300068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.306518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.308790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:33.322652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:33.323492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 279/572] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:33.325163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.364103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a911d90-0ec6-485b-8c7a-92f34dc1f9a5 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.396449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d00993-c7ba-4e6d-9513-0dacc5ba925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.397241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d00993-c7ba-4e6d-9513-0dacc5ba925d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.397521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d00993-c7ba-4e6d-9513-0dacc5ba925d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.397815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d00993-c7ba-4e6d-9513-0dacc5ba925d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.407923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d00993-c7ba-4e6d-9513-0dacc5ba925d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-64d00993-c7ba-4e6d-9513-0dacc5ba925d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:33.415590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:33.416494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:33.417494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 294/573] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:33.418848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d00993-c7ba-4e6d-9513-0dacc5ba925d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.419680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded419ee-0544-4e1a-9402-c14e2fc44f6c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.452418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.453073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.453358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.453635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.454456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:33.454763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.456195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.467674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 280/574] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:33.468447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.512915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d00993-c7ba-4e6d-9513-0dacc5ba925d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.545898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.546694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.546980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.547261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.557446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.559134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.569262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe659-877a-4944-a823-bd49967c5a2d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:33.575338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:33.576360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:33.577283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:33.577283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:33.577283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:33.577283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:33.577283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 295/575] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:33.577692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.605358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1a0c8-2e1c-40fd-adb1-d1742d72a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.606179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1a0c8-2e1c-40fd-adb1-d1742d72a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.606466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1a0c8-2e1c-40fd-adb1-d1742d72a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.606744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1a0c8-2e1c-40fd-adb1-d1742d72a0e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.616445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1a0c8-2e1c-40fd-adb1-d1742d72a0e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5bc1a0c8-2e1c-40fd-adb1-d1742d72a0e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:33.624183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:33.625261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 281/576] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:33.630926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1a0c8-2e1c-40fd-adb1-d1742d72a0e7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.672430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40655e-f2a8-4708-89bf-73c33d15fa29 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.704903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.705556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.705868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.706149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.706930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:33.707244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.708669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.719114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1a0c8-2e1c-40fd-adb1-d1742d72a0e7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.719966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 296/577] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:33.720719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.751257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.751930 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.752218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.752498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.753241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:33.753552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.755002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.762688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 282/578] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:33.763474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.815848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e58886-02ac-4b47-be40-a4c6acd9363c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.847882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec9002-47b2-4366-91e4-3433674334f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.848759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec9002-47b2-4366-91e4-3433674334f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.849048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec9002-47b2-4366-91e4-3433674334f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.849323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec9002-47b2-4366-91e4-3433674334f2 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/3f94e52e32324d8cbf38fb05dc38ea96/roles/3445c3dc3ab4448aaa19f3d511a884b8` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.883338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec9002-47b2-4366-91e4-3433674334f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=3f94e52e32324d8cbf38fb05dc38ea96, role_id=3445c3dc3ab4448aaa19f3d511a884b8)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.884805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec9002-47b2-4366-91e4-3433674334f2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.902105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109fbf9-89d5-44ed-9507-ebf8c412f154 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:33.932654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:33.933556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:33.933877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:33.934155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:33.934860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:33.935163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:33.936509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:33.947874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 283/579] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:33.948734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:33.973449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 297/580] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:04:33 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/3f94e52e32324d8cbf38fb05dc38ea96/roles/3445c3dc3ab4448aaa19f3d511a884b8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:04:33.974340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec9002-47b2-4366-91e4-3433674334f2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.052848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6675a-8a4b-4b7f-909d-90dd2c739631 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.079052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec9002-47b2-4366-91e4-3433674334f2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.089677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.090582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.090857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.091137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.091899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:34.092207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.093606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.102284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 284/581] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:34.103127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.120508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.121186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.121473 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.121751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.122633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=96f6d41c529444fba00bb09ac4224eaa) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:34.122953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.124432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.133606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 298/582] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:33 2022] GET /identity/v3/role_assignments?user.id=96f6d41c529444fba00bb09ac4224eaa&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:34.134668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.221978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab090b3-cdf8-4a3f-80c0-e4991b9bebea None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.253815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e807e-8408-4c53-a074-e406b07bdf0b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.259473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.260688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.261196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.261691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.273612 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.276115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.291336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.292157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.292439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.292711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:34.297126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:34.298137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:34.299095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:34.299095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:34.299095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:34.299095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:34.301210 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.305949 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.306154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 285/583] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:34.307745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:34.323305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:34.324528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:04:34.325678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:34.327974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 299/584] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:34.333319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.413834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0671640-39ec-4ea1-a792-fb4496ec9ce7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.434853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023b768-df79-4405-8278-d28708eb3fee None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.448388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.449295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.449913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.450092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.460924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.462549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.470780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.471659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.472090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.472515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.473557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:34.474052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.475713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:34.484107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:34.485191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:34.486305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:34.486305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:34.486305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:34.486305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:34.491491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 300/586] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:34.491737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 286/586] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:34.492671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.496775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.601226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7c09-917b-4ff0-91c8-9a6f21904a48 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.603017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9a08-fe72-43e0-8951-cc5733faf5ad None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.636730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.637774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.637978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.638313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.638461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.638749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.638844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.639027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.639753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:34.639941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:34.640061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.640418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.641448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.642412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.654808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 287/587] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:34.655654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.666679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 301/588] 10.222.0.25 () {56 vars in 1139 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:34.667525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.763101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf768c6-df5b-40cc-9592-34bdff92ddea None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.770955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84483f-97f7-4f53-a7e6-66e552a16cc2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.797638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.798484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.804029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.804029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.806685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.808448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.810219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.811210 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.811591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.811963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.819701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.821270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:34.824076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:34.825130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 288/589] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:34.826233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:34.839635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:34.841543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:04:34.843638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:34.855495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 302/590] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:34.855495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:34.931866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac09723-72da-40c0-b441-88e1ab0569dc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.953184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e5be-0377-4e6e-a2d1-df9268380374 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:34.966732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.967533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.967814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.968091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.978291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.979826 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.988208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:34.988887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:34.989231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:34.989511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:34.990423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:34.990734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:34.992274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:34.998177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:35.000209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:35.002327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:35.002327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:35.002327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:35.002327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:35.002327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 289/591] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:35.002327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.011484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 303/592] 10.222.0.25 () {56 vars in 1137 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:35.012383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.108193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122d437-71e3-43d5-8349-458ef8c4fdb7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.122270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20eeba-cd2b-4c2f-85b8-2c5292bceb1a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.141972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.142975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.142975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.143322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.144036 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:35.144350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.149423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.160841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.161576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.161906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.162188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.162992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:35.163302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.167681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.171809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 290/593] 10.222.0.25 () {56 vars in 1137 bytes} [Fri May 13 13:04:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:35.172664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.187715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 304/594] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:35.189049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.279033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188c648-7f88-4485-83c8-9c70de525b18 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.306381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dfd21-7b15-4357-b0f1-e1900425f752 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.315469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.316274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.316559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.316835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.327894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.329363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.341917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.342722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.343009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.343286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:35.350219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:35.352191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:35.354813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:35.354813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:35.354813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:35.354813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:35.354813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 291/595] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:35.354813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.355589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.361866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:35.373364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:35.375416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:35.377508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:35.377508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:35.377508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:35.377508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:35.385816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 305/596] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:35.385816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.453633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22246148-6e51-40ac-a488-33c6b81cda86 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.489807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489a6e0-4c28-466e-970c-44799feab188 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.490731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.491534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.491822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.492094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.502286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.503777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:35.518099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:35.518981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:35.519868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:35.519868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:35.519868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:35.519868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:35.519868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:35.519868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:35.519868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:35.519868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:35.520488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.520488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.520488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.520488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.523622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:35.523622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.523982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.524134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 292/597] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:35.524876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.542739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 306/598] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:35.544271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.627969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add0a3e-8961-4faf-8d0f-727c6d2f0afb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.643856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b6d1b-486b-4e44-bc8f-430e0a9a9fc1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.662108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.662779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.663065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.663342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.664128 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:35.664441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.665891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.678992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.679272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 293/599] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:35.679640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.679873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.680041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.680203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.680958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:35.681265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.687234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.699633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 307/600] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:35.700461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.785488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e92cc9-5937-4099-be6c-33432da2f19c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.799027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a9aa-238c-4748-b6bb-e360d6965328 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.821227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.822177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.822498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.822779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.830297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.832081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.833768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:35.834593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:35.838569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:35.838569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:35.842847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:35.844328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:35.847584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:35.849658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 294/601] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:35.851830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:35.859469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:35.861622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 308/602] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:35.863962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:35.968951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3696150-c66d-44b4-bf49-a6ec69cb0828 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:35.969945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b969e0-3d3e-4404-9217-c2e17ac3575b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:36.009686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e678f3f-2d38-436d-982d-63e5a35fa2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:36.011205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678f3f-2d38-436d-982d-63e5a35fa2a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.011490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678f3f-2d38-436d-982d-63e5a35fa2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.011770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e678f3f-2d38-436d-982d-63e5a35fa2a6 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/8ddb0dcaa5304016af1bc1f88845fb80/roles/3445c3dc3ab4448aaa19f3d511a884b8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.016569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:36.017231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.017518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.017810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.018621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:36.018932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:36.020368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:36.027066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e678f3f-2d38-436d-982d-63e5a35fa2a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=8ddb0dcaa5304016af1bc1f88845fb80, role_id=3445c3dc3ab4448aaa19f3d511a884b8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:36.028521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e678f3f-2d38-436d-982d-63e5a35fa2a6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:36.032469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 309/603] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:36.033424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:36.120288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 295/604] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:04:35 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/8ddb0dcaa5304016af1bc1f88845fb80/roles/3445c3dc3ab4448aaa19f3d511a884b8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:04:36.122072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e678f3f-2d38-436d-982d-63e5a35fa2a6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:36.158710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43185ae2-e0ed-4eb3-b52d-10178a44b212 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:36.252190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:36.252896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.253181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.253460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.254320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:36.254633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:36.256159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:36.260530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e678f3f-2d38-436d-982d-63e5a35fa2a6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:36.289503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 310/605] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:36.295469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:36.303625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3667cb-e628-452b-8ce3-4cea4a14bddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:36.304543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3667cb-e628-452b-8ce3-4cea4a14bddc None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.304829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3667cb-e628-452b-8ce3-4cea4a14bddc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.305107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3667cb-e628-452b-8ce3-4cea4a14bddc None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/be3f786b791b4c8796d251948917ca93/roles/3445c3dc3ab4448aaa19f3d511a884b8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.336515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3667cb-e628-452b-8ce3-4cea4a14bddc None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=be3f786b791b4c8796d251948917ca93, role_id=3445c3dc3ab4448aaa19f3d511a884b8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:36.339122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3667cb-e628-452b-8ce3-4cea4a14bddc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:36.426512 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22138-e42c-40ce-9e2d-f2fba1458f20 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:36.445948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 296/606] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:04:36 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/be3f786b791b4c8796d251948917ca93/roles/3445c3dc3ab4448aaa19f3d511a884b8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:04:36.450273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039876f9-3da1-4a67-b11a-5c6ba3a4ed06 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.450273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039876f9-3da1-4a67-b11a-5c6ba3a4ed06 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.450273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039876f9-3da1-4a67-b11a-5c6ba3a4ed06 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.450273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 297/607] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:36.451402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9527-d2fc-4105-95af-8ad6a04b46b8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.451511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9527-d2fc-4105-95af-8ad6a04b46b8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.451818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9527-d2fc-4105-95af-8ad6a04b46b8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.453232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 298/608] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:36.456326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac982ee1-0b8a-43a6-8707-b24a08fee89c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.456618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac982ee1-0b8a-43a6-8707-b24a08fee89c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.456904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac982ee1-0b8a-43a6-8707-b24a08fee89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.468425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08ad76-cb0e-441c-bff0-2c494745bb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:36.469995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08ad76-cb0e-441c-bff0-2c494745bb42 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.470314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08ad76-cb0e-441c-bff0-2c494745bb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.470599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08ad76-cb0e-441c-bff0-2c494745bb42 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/a7955ca85d92485d91abfa401b63f26e/roles/3445c3dc3ab4448aaa19f3d511a884b8` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.482440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ac982ee1-0b8a-43a6-8707-b24a08fee89c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:36.484175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ac982ee1-0b8a-43a6-8707-b24a08fee89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:36.494896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08ad76-cb0e-441c-bff0-2c494745bb42 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=a7955ca85d92485d91abfa401b63f26e, role_id=3445c3dc3ab4448aaa19f3d511a884b8)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:36.496547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08ad76-cb0e-441c-bff0-2c494745bb42 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:36.556171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 299/609] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:36.558581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0cdff-da33-445f-8685-ef4d5cac8923 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.558872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0cdff-da33-445f-8685-ef4d5cac8923 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.559151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0cdff-da33-445f-8685-ef4d5cac8923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.587458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 311/610] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:04:36 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/a7955ca85d92485d91abfa401b63f26e/roles/3445c3dc3ab4448aaa19f3d511a884b8 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:04:36.588456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08ad76-cb0e-441c-bff0-2c494745bb42 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:36.589275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-14b0cdff-da33-445f-8685-ef4d5cac8923 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:36.590513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-14b0cdff-da33-445f-8685-ef4d5cac8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:36.725255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 300/611] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:36.748896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0cdff-da33-445f-8685-ef4d5cac8923 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:36.764636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08ad76-cb0e-441c-bff0-2c494745bb42 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:36.805436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d466c9-1310-4bba-a8c4-c6e05d8aec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:36.806420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d466c9-1310-4bba-a8c4-c6e05d8aec91 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.806708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d466c9-1310-4bba-a8c4-c6e05d8aec91 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.807864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d466c9-1310-4bba-a8c4-c6e05d8aec91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.824171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d466c9-1310-4bba-a8c4-c6e05d8aec91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-32d466c9-1310-4bba-a8c4-c6e05d8aec91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:36.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:36.835584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:36.836422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:36.837859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 312/612] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:36.844599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d466c9-1310-4bba-a8c4-c6e05d8aec91 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:36.877376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0cdff-da33-445f-8685-ef4d5cac8923 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:36.921176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77f801-91b9-410e-9872-cb4cb8efd1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:36.922502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77f801-91b9-410e-9872-cb4cb8efd1dd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:36.922926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77f801-91b9-410e-9872-cb4cb8efd1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:36.923324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77f801-91b9-410e-9872-cb4cb8efd1dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:36.936555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77f801-91b9-410e-9872-cb4cb8efd1dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2a77f801-91b9-410e-9872-cb4cb8efd1dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:36.946499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:36.949082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:36.950249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 301/613] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:36.959243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77f801-91b9-410e-9872-cb4cb8efd1dd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:36.971737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d466c9-1310-4bba-a8c4-c6e05d8aec91 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:37.033152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:37.033940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:37.034159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:37.034441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:37.035183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:37.035485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:37.036853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:37.053332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 313/614] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:37.060191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:37.125218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77f801-91b9-410e-9872-cb4cb8efd1dd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:37.184695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:37.185368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:37.185693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:37.185993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:37.186748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:37.187050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:37.190908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:37.207676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 302/615] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:37.213856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:37.240559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44200e60-e612-4094-8f10-0135535b2487 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:37.274867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:37.276226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:37.276562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:37.276903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:37.289098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:37.291040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:37.309643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:37.310709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:37.312181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:37.312181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:37.312181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:37.312181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:37.312181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 314/616] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:37.321398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:37.340036 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef410e-7335-4411-92be-2d647fb38299 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:37.393156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:37.394040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:37.394364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:37.394643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:37.414551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:37.416080 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:37.445821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:37.446971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:37.448314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:37.448314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:37.448314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:37.448314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:37.448314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 303/617] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:37.453550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:37.555457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb85914-0de1-417b-85fc-e1e620d5c3d3 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:37.564525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2757e1c-394a-4dca-9bcc-b28b7f693ac8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:37.615958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:37.615958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:37.615958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:37.615958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:37.616529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:37.616529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:37.617948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:37.627556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:37.628945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:37.629657 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:37.630378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:37.631711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:37.632450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:37.633509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 304/618] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:37.639528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:37.639839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:37.677267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 315/619] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:37.697392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:37.752100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c54f62-13b5-46ba-a284-f864393fdaec None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:37.788305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:37.789239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:37.789517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:37.789811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:37.798874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:37.800358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:37.818473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:37.819447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:37.820398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 305/620] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:37.826311 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:37.938629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dd60f-1aaf-42eb-a89a-bb4d6a51dd89 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:37.975831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:37.976504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:37.976785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:37.977062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:37.978246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:37.978584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:37.980163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:37.989376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be51f75-296b-44b9-b5a1-9f732bf9d721 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:38.000133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 306/621] 10.222.0.25 () {56 vars in 1135 bytes} [Fri May 13 13:04:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:38.020384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:38.069989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:38.070721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.071010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.071293 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.089509 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:38.091016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:38.122816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:38.123822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:38.124899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 316/622] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:04:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:38.131604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:38.168825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394abb-fb85-40e5-8f03-9b7d3dd6176d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:38.207361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:38.208170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.208457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.208750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:38.222239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:38.242813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:38.243775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:38.245240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:38.245240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:38.245240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:38.245240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:38.245240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 307/623] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:38.252686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:38.305742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ae775-238b-41b4-ab1d-51553bcac2c7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:38.348980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:38.350099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.350593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.351108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.352160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:38.352675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:38.354393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:38.374881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 317/624] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:04:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:38.382281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:38.456052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e33d4f-1f23-4540-819e-eede906ff5b7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:38.494199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743b803-aa0d-43fd-b9d6-9b734d24465a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:38.538391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e75f50-0212-4b8a-a5f2-2c72fbdd1fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:38.540124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e75f50-0212-4b8a-a5f2-2c72fbdd1fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.540851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e75f50-0212-4b8a-a5f2-2c72fbdd1fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.542353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e75f50-0212-4b8a-a5f2-2c72fbdd1fe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.554435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:38.555404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.555948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.556428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.557931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:38.558746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:38.560330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:38.587595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 308/625] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:38.592796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:38.712138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e75f50-0212-4b8a-a5f2-2c72fbdd1fe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:38.714168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e75f50-0212-4b8a-a5f2-2c72fbdd1fe4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:38.733756 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611262-d047-470a-a194-7c067fffaf7d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:38.779032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e085c-fde9-4ed3-8f47-aa78dd7fa41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:38.780221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e085c-fde9-4ed3-8f47-aa78dd7fa41d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.780711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e085c-fde9-4ed3-8f47-aa78dd7fa41d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.781202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e085c-fde9-4ed3-8f47-aa78dd7fa41d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.800147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e085c-fde9-4ed3-8f47-aa78dd7fa41d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:38.804225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e085c-fde9-4ed3-8f47-aa78dd7fa41d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-1d9e085c-fde9-4ed3-8f47-aa78dd7fa41d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:38.825678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:38.826674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:38.827585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:38.827585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:38.827585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:38.827585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:38.827585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:38.827585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:38.827585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:38.827585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:38.831503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 309/626] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:38.832848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8591694-6c23-4087-a486-1bbaf66289f4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.833361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8591694-6c23-4087-a486-1bbaf66289f4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.833473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8591694-6c23-4087-a486-1bbaf66289f4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.839623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 310/627] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:38.840433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8591694-6c23-4087-a486-1bbaf66289f4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:38.891715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 318/628] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:38.893360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99f9f-1a67-4057-ad4f-89433b08c66b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.893904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99f9f-1a67-4057-ad4f-89433b08c66b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.894390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99f9f-1a67-4057-ad4f-89433b08c66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.911246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-5ed99f9f-1a67-4057-ad4f-89433b08c66b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:38.912840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5ed99f9f-1a67-4057-ad4f-89433b08c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:38.948211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8591694-6c23-4087-a486-1bbaf66289f4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:38.982113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 319/629] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:38.983302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed99f9f-1a67-4057-ad4f-89433b08c66b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:38.990587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:38.991275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:38.991560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:38.991839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:38.992646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:38.992987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:38.994595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:39.015672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 311/630] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:39.021393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.086605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed99f9f-1a67-4057-ad4f-89433b08c66b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.125036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.126241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.126735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.127233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.134850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a96eea-ae43-4848-aa8b-65e8a92d35e4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.138814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:39.140635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:39.158610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:39.159583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:39.160488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:39.160488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:39.160488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:39.160488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:39.162905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 320/631] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:39.164068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.179621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700c4c5-55ae-47ad-80f0-a081c0ba7cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.180552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c4c5-55ae-47ad-80f0-a081c0ba7cdc None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.180837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c4c5-55ae-47ad-80f0-a081c0ba7cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.181117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700c4c5-55ae-47ad-80f0-a081c0ba7cdc None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/96f6d41c529444fba00bb09ac4224eaa/roles/3445c3dc3ab4448aaa19f3d511a884b8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.202516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700c4c5-55ae-47ad-80f0-a081c0ba7cdc None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=96f6d41c529444fba00bb09ac4224eaa, role_id=3445c3dc3ab4448aaa19f3d511a884b8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:39.204050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700c4c5-55ae-47ad-80f0-a081c0ba7cdc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:39.268843 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a00c66-a413-4cd6-8819-1c38b6b9a9c1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.293813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 312/632] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:04:39 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/96f6d41c529444fba00bb09ac4224eaa/roles/3445c3dc3ab4448aaa19f3d511a884b8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:04:39.293813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700c4c5-55ae-47ad-80f0-a081c0ba7cdc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.315442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4fa5-8b69-463e-ba23-9bfeef111e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.316639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4fa5-8b69-463e-ba23-9bfeef111e92 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.316929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4fa5-8b69-463e-ba23-9bfeef111e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.317400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4fa5-8b69-463e-ba23-9bfeef111e92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.334442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde4fa5-8b69-463e-ba23-9bfeef111e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-afde4fa5-8b69-463e-ba23-9bfeef111e92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:39.345230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:39.346266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:39.347096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 321/633] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:39.350256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b617f0-c10b-45f9-8582-d1804b892e89 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.350563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b617f0-c10b-45f9-8582-d1804b892e89 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.350848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b617f0-c10b-45f9-8582-d1804b892e89 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.352628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 322/634] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:39.355366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2e07a-da4b-436a-b629-6cb4423119cd None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.355664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2e07a-da4b-436a-b629-6cb4423119cd None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.355948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2e07a-da4b-436a-b629-6cb4423119cd None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.357634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 323/635] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:39.358385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2e07a-da4b-436a-b629-6cb4423119cd None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.423806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700c4c5-55ae-47ad-80f0-a081c0ba7cdc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.462057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d77962-2c19-48ef-9b21-71974d2aaef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.462731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d77962-2c19-48ef-9b21-71974d2aaef3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.463522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d77962-2c19-48ef-9b21-71974d2aaef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.463791 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d77962-2c19-48ef-9b21-71974d2aaef3 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.464521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d77962-2c19-48ef-9b21-71974d2aaef3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:39.464828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d77962-2c19-48ef-9b21-71974d2aaef3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:39.466260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d77962-2c19-48ef-9b21-71974d2aaef3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:39.485947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2e07a-da4b-436a-b629-6cb4423119cd None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.486386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 313/636] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:39.487957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7e250-778e-47ef-82f4-ee808576f39f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.488354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7e250-778e-47ef-82f4-ee808576f39f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.488734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7e250-778e-47ef-82f4-ee808576f39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.507865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6ff7e250-778e-47ef-82f4-ee808576f39f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:39.509197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ff7e250-778e-47ef-82f4-ee808576f39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:39.522749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89b983-61d9-4815-ae3c-38ad6886df4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.524081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89b983-61d9-4815-ae3c-38ad6886df4b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.524631 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89b983-61d9-4815-ae3c-38ad6886df4b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.525065 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89b983-61d9-4815-ae3c-38ad6886df4b None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.526659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89b983-61d9-4815-ae3c-38ad6886df4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:39.527179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89b983-61d9-4815-ae3c-38ad6886df4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:39.528740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89b983-61d9-4815-ae3c-38ad6886df4b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:39.537158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 324/637] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:39.538536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0300ec1-f3e6-45a1-8596-d46bcc04ad1c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.538876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0300ec1-f3e6-45a1-8596-d46bcc04ad1c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.539154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0300ec1-f3e6-45a1-8596-d46bcc04ad1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.556416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d0300ec1-f3e6-45a1-8596-d46bcc04ad1c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:39.557594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d0300ec1-f3e6-45a1-8596-d46bcc04ad1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:39.586108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 314/638] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:39.586108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7e250-778e-47ef-82f4-ee808576f39f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.631910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 325/639] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:39.633040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0300ec1-f3e6-45a1-8596-d46bcc04ad1c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.685609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7e250-778e-47ef-82f4-ee808576f39f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.718182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.718973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.719254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.719530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.726055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:39.727496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:39.735014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0300ec1-f3e6-45a1-8596-d46bcc04ad1c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:39.741470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:39.742570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:39.743636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:39.743636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:39.743636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:39.743636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:39.743636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:39.743636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:39.743636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:39.743636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:39.747900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 315/640] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:39.747900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.773569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.774338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.774621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.774897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.784647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:39.786167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:39.804874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:39.805959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:39.807437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:39.807437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:39.807437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:39.807437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:39.807437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 326/641] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:39.807437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.848311 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4aaed-5a00-466f-9968-17bdd68f04d4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.882138 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ec97-91ce-4575-8d06-d62f3c691e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.883333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ec97-91ce-4575-8d06-d62f3c691e21 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.883621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ec97-91ce-4575-8d06-d62f3c691e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.883897 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ec97-91ce-4575-8d06-d62f3c691e21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.895480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161ec97-91ce-4575-8d06-d62f3c691e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d161ec97-91ce-4575-8d06-d62f3c691e21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:39.903742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:39.904738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:39.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 316/642] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:39.906394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ec97-91ce-4575-8d06-d62f3c691e21 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:39.911630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e82c-30ff-421f-ac00-aea75012ae9f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:39.944876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd987c-ef2c-4252-b2a3-ce72b0577f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:39.945674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd987c-ef2c-4252-b2a3-ce72b0577f9c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:39.946330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd987c-ef2c-4252-b2a3-ce72b0577f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:39.946608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd987c-ef2c-4252-b2a3-ce72b0577f9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:39.957476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd987c-ef2c-4252-b2a3-ce72b0577f9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-3efd987c-ef2c-4252-b2a3-ce72b0577f9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:39.965693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:39.966834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:39.967743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 327/643] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:39.968509 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd987c-ef2c-4252-b2a3-ce72b0577f9c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.010773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ec97-91ce-4575-8d06-d62f3c691e21 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.056364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.057151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.057528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.057934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.059073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:40.059480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.061143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.078642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 317/644] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:40.078642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.096021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd987c-ef2c-4252-b2a3-ce72b0577f9c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.131882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.132650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.133031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.133385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.134217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:40.134610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.136062 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.147697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 328/645] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:40.148575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.183701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426f137-e51c-4c98-9c20-3b9cc87bc27b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.215485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.216145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.216424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.216706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.217436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:40.217735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.219113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.231423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 318/646] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:40.231423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.250729 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729acc27-702a-4b19-9292-ab563bc248b4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.284940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.285608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.285922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.286201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.286942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:40.287246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.288608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.300888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 329/647] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:40.300888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.326557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eac0e-ffb0-43c7-ac42-7ad2929af864 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.368485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.369569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.370091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.370557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.372056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=3f94e52e32324d8cbf38fb05dc38ea96) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:40.372604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.374652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.380998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 319/648] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/role_assignments?user.id=3f94e52e32324d8cbf38fb05dc38ea96&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:40.383022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.401735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef201c13-6efa-4425-b646-d933d79e2ad3 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.440146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.441424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.441953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.442438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.451294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.453228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:40.469240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:40.470335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 13 13:04:40.471284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:40.481423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 330/649] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:40.482652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.506394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4532-a62f-4427-8334-b5a09bb4905f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.543941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.544747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.545028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.545301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.557183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.559019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:40.579229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:40.580259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:40.581159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:40.581159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:40.581159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:40.581159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:40.581826 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 320/650] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:40.582335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.595660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973208d-4dba-4db8-a750-99a0d6e81971 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.633729 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.634027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.634188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.634468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.646602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.648075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:40.665878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:40.666933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:40.668026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:40.668026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:40.668026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:40.668026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:40.671767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 331/651] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:40.672616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.704521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547cf3e-ce5f-4820-a2fb-8f297714ae00 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.742342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.743402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.744187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.744877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.745916 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:40.746369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.748615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.771512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 321/652] 10.222.0.25 () {56 vars in 1139 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:40.772542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.788419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd3c23-ba16-4f17-966d-48dbcebd8452 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.831522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.832994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.833429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.833877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.835218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:40.835820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.837444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.852994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 332/653] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:40.854686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.878575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5c40-54ce-4962-90fa-c1969dc37376 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:40.913185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:40.914072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:40.914460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:40.914858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:40.915680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:40.916652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:40.918166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:40.934506 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 322/654] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:40.935505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:40.974536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f4b98-31dc-4548-bced-2ec9c68e1ccb None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.020424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359df34-a218-4573-a002-6936a00da98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.021367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359df34-a218-4573-a002-6936a00da98f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.021855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359df34-a218-4573-a002-6936a00da98f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.022274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359df34-a218-4573-a002-6936a00da98f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.046195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd34179-50db-414a-b208-4128641ad609 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.092596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.093824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.094183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.094468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.095947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359df34-a218-4573-a002-6936a00da98f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.097532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359df34-a218-4573-a002-6936a00da98f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.105199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.107247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:41.126815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:41.127869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:41.128811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 323/655] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:41.129856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:41.208265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 333/656] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:41.210042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359df34-a218-4573-a002-6936a00da98f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:41.250144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34556e6f-a71f-4846-8130-0caa32513c2b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.283677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.284474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.284818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.285096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.293318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.294811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:41.309493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:41.311682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:04:41.313912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:41.317216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 324/657] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:41.317216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:41.326255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359df34-a218-4573-a002-6936a00da98f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.371723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.372976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.373421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.374054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.375455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:41.375968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.377850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.404083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 334/658] 10.222.0.25 () {56 vars in 1137 bytes} [Fri May 13 13:04:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:41.404083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:41.437850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4eb25-705b-472c-82d7-3bb1ad985590 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.488331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.489616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.490497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.491180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.507316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.509181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.514348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c94f47-0713-4a11-b553-56cb19c4ab4f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:41.527336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:41.528254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:41.529135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:41.529135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:41.529135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:41.529135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:41.531391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 325/659] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:41.532520 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:41.554681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.555413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.555706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.555986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.557416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:41.557730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.559473 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.580892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 335/660] 10.222.0.25 () {56 vars in 1137 bytes} [Fri May 13 13:04:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:41.581922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:41.640178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358dd46-9f98-48be-894d-78872a4b034b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.672469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b81da-d4ba-457e-af18-dda751fa87d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.673119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b81da-d4ba-457e-af18-dda751fa87d8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.673406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b81da-d4ba-457e-af18-dda751fa87d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.673687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b81da-d4ba-457e-af18-dda751fa87d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.679584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051f931-d6c4-45be-9ea7-c615ea856cfa None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.714636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820fd4a-fa64-41ce-97d3-2789af69a5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.715304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820fd4a-fa64-41ce-97d3-2789af69a5cd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.715588 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820fd4a-fa64-41ce-97d3-2789af69a5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.715893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820fd4a-fa64-41ce-97d3-2789af69a5cd None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.716621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820fd4a-fa64-41ce-97d3-2789af69a5cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:41.716921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820fd4a-fa64-41ce-97d3-2789af69a5cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.718332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820fd4a-fa64-41ce-97d3-2789af69a5cd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.729419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 336/661] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:41.730739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ecf6c-6b3a-4780-b921-2b6b16cc7354 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.731029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ecf6c-6b3a-4780-b921-2b6b16cc7354 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.731304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ecf6c-6b3a-4780-b921-2b6b16cc7354 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.734662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 337/662] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:41.734662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ecf6c-6b3a-4780-b921-2b6b16cc7354 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:41.750696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b81da-d4ba-457e-af18-dda751fa87d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.752208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b81da-d4ba-457e-af18-dda751fa87d8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.838424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ecf6c-6b3a-4780-b921-2b6b16cc7354 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.868227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 326/663] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:41.868227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b81da-d4ba-457e-af18-dda751fa87d8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:41.877287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842fba3-8a24-467d-af0c-b4dab1b44985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.877978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842fba3-8a24-467d-af0c-b4dab1b44985 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.878263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842fba3-8a24-467d-af0c-b4dab1b44985 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.878542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842fba3-8a24-467d-af0c-b4dab1b44985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:41.942184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842fba3-8a24-467d-af0c-b4dab1b44985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:41.943556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842fba3-8a24-467d-af0c-b4dab1b44985 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:41.961179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b81da-d4ba-457e-af18-dda751fa87d8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:41.993893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeb571-1fde-4f80-a6d7-3f6bc51f8ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:41.994705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeb571-1fde-4f80-a6d7-3f6bc51f8ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:41.994997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeb571-1fde-4f80-a6d7-3f6bc51f8ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:41.995276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeb571-1fde-4f80-a6d7-3f6bc51f8ee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.001738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adeb571-1fde-4f80-a6d7-3f6bc51f8ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.003216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adeb571-1fde-4f80-a6d7-3f6bc51f8ee6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-9adeb571-1fde-4f80-a6d7-3f6bc51f8ee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:42.016919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:42.017974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:42.018986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 327/664] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:42.019592 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec37e19-b905-4f9e-b64a-de6e29d506ab None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.019880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec37e19-b905-4f9e-b64a-de6e29d506ab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.020157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec37e19-b905-4f9e-b64a-de6e29d506ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.036437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-aec37e19-b905-4f9e-b64a-de6e29d506ab None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:42.037636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aec37e19-b905-4f9e-b64a-de6e29d506ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:42.044539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 338/665] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:42.045709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842fba3-8a24-467d-af0c-b4dab1b44985 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.114606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 328/666] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:42.115768 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec37e19-b905-4f9e-b64a-de6e29d506ab None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.160320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842fba3-8a24-467d-af0c-b4dab1b44985 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.194925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.195649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.195935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.196205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.206028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.207493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.213711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec37e19-b905-4f9e-b64a-de6e29d506ab None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:42.224595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:42.225517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:42.227261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:42.227261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:42.227261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:42.227261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:42.227261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 339/667] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:42.227261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.257921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.259403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.260670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.261185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.262212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:42.262727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.264350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.279887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 329/668] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:42.281671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.324296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a1a48-ef38-405b-8205-4195db99c184 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.360747 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.361905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.362359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.362790 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.373426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.375081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.380227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961fdaa-b65f-4d06-af34-7b4d444b3c61 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:42.396738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:42.397776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:42.399044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:42.399044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:42.399044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:42.399044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:42.405627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 340/669] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:42.405627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.414222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c393-fab8-4604-9866-f84fb966b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.415025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c393-fab8-4604-9866-f84fb966b135 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.415314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c393-fab8-4604-9866-f84fb966b135 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.415594 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c393-fab8-4604-9866-f84fb966b135 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.425692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6c393-fab8-4604-9866-f84fb966b135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-21e6c393-fab8-4604-9866-f84fb966b135 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:42.434437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:42.435348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:42.436201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 330/670] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:42.436901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c393-fab8-4604-9866-f84fb966b135 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.503260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1bab9-3265-48b8-a2a7-0196f1e1361e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.534072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c393-fab8-4604-9866-f84fb966b135 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.535852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.536519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.536809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.537087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.537827 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:42.538140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.539523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.554920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 341/671] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:42.554920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.568768 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.569421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.569706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.570012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.570955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=8ddb0dcaa5304016af1bc1f88845fb80) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:42.571264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.573289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.579000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 331/672] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/role_assignments?user.id=8ddb0dcaa5304016af1bc1f88845fb80&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:42.579782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.668961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc534d4-4aeb-4dd7-9abc-169790e5263a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.691847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734011f-9d9c-468f-863c-6287ede8b122 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.709364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.710517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.711006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.711531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.712525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:42.713125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.714786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.729216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 342/673] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:42.730614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.733759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.734434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.734726 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.735012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.736415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:42.736415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.737513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.747418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 332/674] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:42.748350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.858045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9e723-740d-49fa-9472-c65583e580a8 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.861414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc0330-c9fb-4740-9611-38c943c33c47 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:42.892878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.893761 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.894138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.894473 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.896271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbf707-5fd4-48e3-b46d-2c720917ad80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:42.897141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf707-5fd4-48e3-b46d-2c720917ad80 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.897494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf707-5fd4-48e3-b46d-2c720917ad80 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.897867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf707-5fd4-48e3-b46d-2c720917ad80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.901808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.903320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.905590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbf707-5fd4-48e3-b46d-2c720917ad80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:42.907056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbf707-5fd4-48e3-b46d-2c720917ad80 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:42.916989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:42.918082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:42.919170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:42.919170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:42.919170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:42.919170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:42.919170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:42.919170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:42.919170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:42.919170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:42.925394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 343/675] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-76dbf707-5fd4-48e3-b46d-2c720917ad80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:42.925546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:42.926445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:42.927743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:42.927743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:42.927743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:42.927743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:42.927743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:42.927743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:42.927743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:42.927743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:42.928179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:42.928257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 333/676] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:42.928257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116165-b595-4811-9ff0-87319822b298 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:42.928257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116165-b595-4811-9ff0-87319822b298 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:42.928257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116165-b595-4811-9ff0-87319822b298 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:42.928257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 334/677] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:42.928257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad116165-b595-4811-9ff0-87319822b298 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.026655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad116165-b595-4811-9ff0-87319822b298 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.028623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391223f-c91d-4da4-9882-0ae63b04398a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.064268 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212009af-4991-43cd-a5ce-b52039b1755e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.065038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212009af-4991-43cd-a5ce-b52039b1755e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.065399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212009af-4991-43cd-a5ce-b52039b1755e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.065754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212009af-4991-43cd-a5ce-b52039b1755e None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.066647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212009af-4991-43cd-a5ce-b52039b1755e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:43.067029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212009af-4991-43cd-a5ce-b52039b1755e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.067894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.068469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212009af-4991-43cd-a5ce-b52039b1755e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.069020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.069445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.069908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.084918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 335/678] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:43.085100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.085275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce086a-79e2-4ac0-9235-277a2f8d0e4d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.085275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce086a-79e2-4ac0-9235-277a2f8d0e4d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.085275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce086a-79e2-4ac0-9235-277a2f8d0e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.085493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.100321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f5ce086a-79e2-4ac0-9235-277a2f8d0e4d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:43.101591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f5ce086a-79e2-4ac0-9235-277a2f8d0e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:43.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:43.104273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:43.105343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:43.105343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:43.105343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:43.105343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:43.110517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 344/679] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:43.110517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.179462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 336/680] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:43.180139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce086a-79e2-4ac0-9235-277a2f8d0e4d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.233051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0589cc-2bab-4530-9e77-b5cb46307c5b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.266427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.267095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.267406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.267661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.268456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:43.268766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.270222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.276441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce086a-79e2-4ac0-9235-277a2f8d0e4d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.286921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 345/681] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:43.286921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.310404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.311070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.311360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.311657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.312489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=be3f786b791b4c8796d251948917ca93) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:43.312804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.314344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.320676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 337/682] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/role_assignments?user.id=be3f786b791b4c8796d251948917ca93&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:43.321451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.424649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80bfa9-ee23-4905-9d2f-db95cabdf9ba None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.435956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e80ea-9872-453d-a58d-1db2490d76f8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.470278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.471511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.471943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.472424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.472873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c6e36-2742-455c-956a-5c5f540e7674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.473501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:43.473895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c6e36-2742-455c-956a-5c5f540e7674 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.474146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.474604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c6e36-2742-455c-956a-5c5f540e7674 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.475193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c6e36-2742-455c-956a-5c5f540e7674 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.476156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.488471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c6e36-2742-455c-956a-5c5f540e7674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:43.495113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 346/683] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:43.496351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8d1c6e36-2742-455c-956a-5c5f540e7674 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:43.499147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:43.500241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:43.501239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:43.504031 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 338/684] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:43.504326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c6e36-2742-455c-956a-5c5f540e7674 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.605346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7adbc9-563c-4fa7-a2ed-28f6113d726d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.607898 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c6e36-2742-455c-956a-5c5f540e7674 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.640710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.641240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.641755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.642058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.642345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.642514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.642612 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.642994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.644112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=a7955ca85d92485d91abfa401b63f26e) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:43.644566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.646148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.650241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.651688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.662263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 347/685] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/role_assignments?user.id=a7955ca85d92485d91abfa401b63f26e&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:43.663489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:43.666598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:43.667699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:43.668818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 339/686] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:43.673759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.780335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697147b-26c7-4f7a-be4c-0ad06b3b9021 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.817049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.818087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.818514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.818962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.819868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:43.820317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.821880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.831669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 348/687] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:43.839664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9743a-2794-4c0f-bc71-331c2eac7878 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:43.840814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:43.882127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:43.882784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:43.883069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:43.883550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:43.884415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:43.885276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:43.888457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:43.920392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 340/688] 10.222.0.25 () {56 vars in 1135 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:43.926120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.009622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4983c3-3380-458d-821f-fbf804228e54 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:44.046691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:44.048170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:44.048795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:44.049295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b89e7-6f51-41f0-8537-34df86aa6b34 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:44.049671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:44.069309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:44.071825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:44.090986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef13e-576d-461c-bab8-8d7a9ab6ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:44.092462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef13e-576d-461c-bab8-8d7a9ab6ac08 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:44.094195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef13e-576d-461c-bab8-8d7a9ab6ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:44.094639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef13e-576d-461c-bab8-8d7a9ab6ac08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:44.105443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:44.106585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:44.107694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:44.107694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:44.107694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:44.107694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:44.109199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 349/689] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:44.114495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.182343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cef13e-576d-461c-bab8-8d7a9ab6ac08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:44.184133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cef13e-576d-461c-bab8-8d7a9ab6ac08 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:44.225534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2830f1-37c5-4d9d-bda0-4779f0a1165c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:44.259612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:44.260989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:44.261418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:44.261884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:44.262761 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:44.263216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:44.264757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:44.281949 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 350/690] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:44.283666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.284476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 341/691] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:04:44.290473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef13e-576d-461c-bab8-8d7a9ab6ac08 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.385732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066d0b2-5000-4d58-a32a-b259eab613d4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:44.389038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef13e-576d-461c-bab8-8d7a9ab6ac08 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:44.420575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:44.421754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:44.422286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:44.422739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:44.423287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:44.424171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:44.424550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:44.424917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:44.430894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:44.432606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:44.435904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:44.437440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:44.448568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:44.449710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:44.450894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:44.453107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 351/692] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:04:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:44.458624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:44.459394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:44.460422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:44.461367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:44.461367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:44.461367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:44.461367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:44.462186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 342/693] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:44.470932 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.562164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e618a-ce2a-4b35-8212-e71f481f767e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:44.598219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:44.599269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:44.599778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:44.600279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:44.601323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:44.601890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:44.604063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:44.623417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 352/694] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:04:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:44.629147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.629972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9a4fa-c97d-49a3-9876-e7804bd8c591 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:44.673278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:44.673985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:44.674266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:44.674539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:44.675314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:44.675616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:44.677049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:44.708313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 343/695] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:44.718851 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.752208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09bc7b-5d6b-4da0-9ef3-c847d0fdfea2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:44.786877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a948c-0ec7-4587-b00d-3c8e960dd288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:44.787889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a948c-0ec7-4587-b00d-3c8e960dd288 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:44.788407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a948c-0ec7-4587-b00d-3c8e960dd288 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:44.788886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a948c-0ec7-4587-b00d-3c8e960dd288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:44.859167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a948c-0ec7-4587-b00d-3c8e960dd288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:44.861005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a948c-0ec7-4587-b00d-3c8e960dd288 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:44.964613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 353/696] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 13 13:04:44.970253 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a948c-0ec7-4587-b00d-3c8e960dd288 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:44.975702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4075-128f-451c-8046-03cfc752bc0d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.028187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.028992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.029278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.029615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.044011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.045472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:45.070394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:45.071333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:45.072255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 344/697] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:45.081065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.109240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a948c-0ec7-4587-b00d-3c8e960dd288 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.144673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514533a6-8736-4d1f-ae50-e5831303ee2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.145508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514533a6-8736-4d1f-ae50-e5831303ee2d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.145757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514533a6-8736-4d1f-ae50-e5831303ee2d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.146071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514533a6-8736-4d1f-ae50-e5831303ee2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.156770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514533a6-8736-4d1f-ae50-e5831303ee2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.158258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514533a6-8736-4d1f-ae50-e5831303ee2d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-514533a6-8736-4d1f-ae50-e5831303ee2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:45.175564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:45.177725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:45.180040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:45.180040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:45.180040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:45.180040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:45.180040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 354/698] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:45.180040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052308a-6c22-495c-a76e-acb52a1c13ce None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.180040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052308a-6c22-495c-a76e-acb52a1c13ce None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.180040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052308a-6c22-495c-a76e-acb52a1c13ce None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.184796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 355/699] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:45.186318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee708-f813-406e-863a-71bcb6892fad None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.188599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee708-f813-406e-863a-71bcb6892fad None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.188599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee708-f813-406e-863a-71bcb6892fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.203951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e8aee708-f813-406e-863a-71bcb6892fad None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:45.205186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e8aee708-f813-406e-863a-71bcb6892fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:45.214753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad5088-fd6a-440b-a0d2-1e65aa63e0f3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.249140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.249896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.250185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.250465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.251265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:45.251577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.253015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.265605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 345/700] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:45.266578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.274659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 356/701] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:45.282325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aee708-f813-406e-863a-71bcb6892fad None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.365261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9cd85-c89d-4773-a3ed-7612ad3b0131 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.394639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aee708-f813-406e-863a-71bcb6892fad None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.398757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.399425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.399711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.399991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.401280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:45.401589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.403018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.415330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 346/702] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:45.416248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.430499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.431847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.432126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.432766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.434148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=96f6d41c529444fba00bb09ac4224eaa) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:45.434459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.436321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.443994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 357/703] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/role_assignments?user.id=96f6d41c529444fba00bb09ac4224eaa&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:45.445116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.535207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dea895-ee4f-40de-b3bf-3c571d5dd216 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.583650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a4c26-3b6a-45cb-8bdf-c68c0b6c513e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.584627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.586812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.587280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.589103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.589967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.591561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:45.618607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:45.619923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:45.620894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:45.633149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 347/704] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:45.638887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.659701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.660871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.661351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.661773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.670502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.672860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:45.689257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:45.690294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:45.691303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:45.691303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:45.691303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:45.691303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:45.691303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:45.691303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:45.691303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:45.691303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:45.693901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 358/705] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:45.699935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.794091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0855ad6-1fd4-474e-aad6-26168fd455e3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.835681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.836891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.837489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.838278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.839416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] RBAC: Adding query filter params (name=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:45.839890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.840785 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d623c-0ce6-4a02-9cda-bbc1059d38d2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:45.841600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.848070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 348/706] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:45.854733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.883903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:45.885095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:45.885381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:45.885657 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:45.886930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:45.887250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:45.889119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:45.905082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 359/707] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:45.910763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:45.980301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406c9a0-80df-42ec-8632-8a0fd917756d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:46.017647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07600f2b-3397-4423-b20b-94750628dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:46.018452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600f2b-3397-4423-b20b-94750628dc58 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.018672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600f2b-3397-4423-b20b-94750628dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.018956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600f2b-3397-4423-b20b-94750628dc58 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.019660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07600f2b-3397-4423-b20b-94750628dc58 None admin] RBAC: Adding query filter params (type=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:46.019974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07600f2b-3397-4423-b20b-94750628dc58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:46.021430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07600f2b-3397-4423-b20b-94750628dc58 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:46.027585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 349/708] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:46.028980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658146c7-a1e2-46d4-9d3d-d1a82c20938a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.029272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658146c7-a1e2-46d4-9d3d-d1a82c20938a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.029550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658146c7-a1e2-46d4-9d3d-d1a82c20938a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.031141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 350/709] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:46.037103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bece8a4-1adb-46ef-be20-7aaa2fc38d6a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.037410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bece8a4-1adb-46ef-be20-7aaa2fc38d6a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.037727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bece8a4-1adb-46ef-be20-7aaa2fc38d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.038866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f475e6c-1b6d-4e8e-8e9b-1f592e36fbbc None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:46.056205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7bece8a4-1adb-46ef-be20-7aaa2fc38d6a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:46.057485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7bece8a4-1adb-46ef-be20-7aaa2fc38d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:46.085665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:46.088355 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.088681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.088972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.090602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=3f94e52e32324d8cbf38fb05dc38ea96) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:46.091790 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:46.096066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:46.112042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 360/710] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:45 2022] GET /identity/v3/role_assignments?user.id=3f94e52e32324d8cbf38fb05dc38ea96&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=663341777b0b44128ed78fe8d40f6462 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:46.156561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 351/711] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:46.182358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:46.296906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdd116-3834-4b9e-a492-ddb511f6c01f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:46.332241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:46.332985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.333272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.333552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.345876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:46.347441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:46.366014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:46.366984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:46.369537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:46.369537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:46.369537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:46.369537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:46.369537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 361/712] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:46.374073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bece8a4-1adb-46ef-be20-7aaa2fc38d6a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:46.477136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bece8a4-1adb-46ef-be20-7aaa2fc38d6a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:46.516887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d598a-e0a1-46e6-ac74-690158b75e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:46.517993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d598a-e0a1-46e6-ac74-690158b75e42 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.518515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d598a-e0a1-46e6-ac74-690158b75e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.519006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d598a-e0a1-46e6-ac74-690158b75e42 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.519968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d598a-e0a1-46e6-ac74-690158b75e42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:46.520489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d598a-e0a1-46e6-ac74-690158b75e42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:46.522113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d598a-e0a1-46e6-ac74-690158b75e42 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:46.539021 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 352/713] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:46.544660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:46.701741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e9cc2-3701-4ed5-9fa0-ecf01cf02019 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:46.746381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333382f-8f6a-49b8-b59e-54b5005a01da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:46.747142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333382f-8f6a-49b8-b59e-54b5005a01da None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.747433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333382f-8f6a-49b8-b59e-54b5005a01da None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.747715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333382f-8f6a-49b8-b59e-54b5005a01da None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.748442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333382f-8f6a-49b8-b59e-54b5005a01da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:46.749980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333382f-8f6a-49b8-b59e-54b5005a01da None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:46.840447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 362/714] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:04:46 2022] POST /identity/v3/users => generated 254 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:46.945445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30346-6c03-4c4c-ba74-a8a5235d7fc8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.945811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30346-6c03-4c4c-ba74-a8a5235d7fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.946034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd30346-6c03-4c4c-ba74-a8a5235d7fc8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.947717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 353/715] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:46.953404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d58a1e-9a6b-41dc-86b9-3c5e49cdc64d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:46.954023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d58a1e-9a6b-41dc-86b9-3c5e49cdc64d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:46.954521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d58a1e-9a6b-41dc-86b9-3c5e49cdc64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:46.973012 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e7d58a1e-9a6b-41dc-86b9-3c5e49cdc64d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:46.974574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e7d58a1e-9a6b-41dc-86b9-3c5e49cdc64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:47.046766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 363/716] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:47.070222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd30346-6c03-4c4c-ba74-a8a5235d7fc8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:47.210097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd30346-6c03-4c4c-ba74-a8a5235d7fc8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:47.248953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:47.249764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:47.250073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:47.250344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:47.250992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:47.252353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:47.264769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:47.265740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:47.266671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 354/717] 10.222.0.25 () {56 vars in 1129 bytes} [Fri May 13 13:04:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:47.272531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d58a1e-9a6b-41dc-86b9-3c5e49cdc64d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:47.381745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d58a1e-9a6b-41dc-86b9-3c5e49cdc64d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:47.417423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b459-0acb-45ef-86c3-fee1412e9dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:47.418364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b459-0acb-45ef-86c3-fee1412e9dce None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:47.418812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b459-0acb-45ef-86c3-fee1412e9dce None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:47.419236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b459-0acb-45ef-86c3-fee1412e9dce None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:47.420080 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254b459-0acb-45ef-86c3-fee1412e9dce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:47.420532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254b459-0acb-45ef-86c3-fee1412e9dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:47.422078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254b459-0acb-45ef-86c3-fee1412e9dce None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:47.426986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 364/718] 10.222.0.25 () {56 vars in 1124 bytes} [Fri May 13 13:04:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:47.432767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:47.582078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ccaef-68c1-412c-9196-7bdf89b75a5b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:47.626939 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579c6e-044c-4070-9563-6722a55642b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:47.627742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579c6e-044c-4070-9563-6722a55642b7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:47.627742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579c6e-044c-4070-9563-6722a55642b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:47.628612 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579c6e-044c-4070-9563-6722a55642b7 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:47.629669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85579c6e-044c-4070-9563-6722a55642b7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:47.629997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85579c6e-044c-4070-9563-6722a55642b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:47.631389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85579c6e-044c-4070-9563-6722a55642b7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:47.641748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 355/719] 10.222.0.25 () {56 vars in 1124 bytes} [Fri May 13 13:04:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:48.253304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42f7ea-75a4-49f9-8cf7-a7b207787e1a None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:48.254097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42f7ea-75a4-49f9-8cf7-a7b207787e1a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:48.254612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42f7ea-75a4-49f9-8cf7-a7b207787e1a None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:48.256451 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 365/720] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:48.261686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053da531-66b7-42ac-ad62-0746dc6af1cd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:48.262005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053da531-66b7-42ac-ad62-0746dc6af1cd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:48.262295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053da531-66b7-42ac-ad62-0746dc6af1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:48.286812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-053da531-66b7-42ac-ad62-0746dc6af1cd None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:48.288044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-053da531-66b7-42ac-ad62-0746dc6af1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:48.372736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 356/721] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:48.411349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42f7ea-75a4-49f9-8cf7-a7b207787e1a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:48.547549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42f7ea-75a4-49f9-8cf7-a7b207787e1a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:48.593528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c8cfb-f852-4e97-bbbf-c239b3d18c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:48.594373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c8cfb-f852-4e97-bbbf-c239b3d18c9a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:48.594659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c8cfb-f852-4e97-bbbf-c239b3d18c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:48.594937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c8cfb-f852-4e97-bbbf-c239b3d18c9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:48.611894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c8cfb-f852-4e97-bbbf-c239b3d18c9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-420c8cfb-f852-4e97-bbbf-c239b3d18c9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:48.620144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:48.621067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:48.623995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 366/722] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:48.639001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053da531-66b7-42ac-ad62-0746dc6af1cd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:48.763111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053da531-66b7-42ac-ad62-0746dc6af1cd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:48.805344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e14b9a-6a76-4a95-9bc7-c4d1872fa1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:48.805986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14b9a-6a76-4a95-9bc7-c4d1872fa1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:48.806301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14b9a-6a76-4a95-9bc7-c4d1872fa1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:48.806583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14b9a-6a76-4a95-9bc7-c4d1872fa1e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:48.807354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e14b9a-6a76-4a95-9bc7-c4d1872fa1e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:48.807667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e14b9a-6a76-4a95-9bc7-c4d1872fa1e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:48.809359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e14b9a-6a76-4a95-9bc7-c4d1872fa1e6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:48.819058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 357/723] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:48.825553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c8cfb-f852-4e97-bbbf-c239b3d18c9a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:48.926106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46511b-7664-4ba0-92aa-a7e7c8bb982f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:48.926468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46511b-7664-4ba0-92aa-a7e7c8bb982f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:48.926674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46511b-7664-4ba0-92aa-a7e7c8bb982f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:48.928320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 358/724] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:48.934598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0e49-77c1-4f14-9924-9844d3011c77 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:48.934873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0e49-77c1-4f14-9924-9844d3011c77 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:48.935159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b0e49-77c1-4f14-9924-9844d3011c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:48.967339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0b3b0e49-77c1-4f14-9924-9844d3011c77 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:48.968559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0b3b0e49-77c1-4f14-9924-9844d3011c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:48.974886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c8cfb-f852-4e97-bbbf-c239b3d18c9a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:49.020075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a72b9-53e2-4b17-93d5-32d839c5fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:49.021260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a72b9-53e2-4b17-93d5-32d839c5fe4f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.021680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a72b9-53e2-4b17-93d5-32d839c5fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.023860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a72b9-53e2-4b17-93d5-32d839c5fe4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.034817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a72b9-53e2-4b17-93d5-32d839c5fe4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:49.036625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a72b9-53e2-4b17-93d5-32d839c5fe4f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-437a72b9-53e2-4b17-93d5-32d839c5fe4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:49.054683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:49.055675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:49.057151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:49.057151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:49.057151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:49.057151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:49.064448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 367/725] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:49.064448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9dfa0-50f7-4d44-84c7-6502697f6874 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.064448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9dfa0-50f7-4d44-84c7-6502697f6874 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.064448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9dfa0-50f7-4d44-84c7-6502697f6874 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.065236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 368/726] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:49.068027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9dfa0-50f7-4d44-84c7-6502697f6874 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:49.076657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 359/727] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 13 13:04:49.077994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d5fe-c45a-47df-a307-30e2c6f3fc24 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.078285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d5fe-c45a-47df-a307-30e2c6f3fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.078566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01d5fe-c45a-47df-a307-30e2c6f3fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.109621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bc01d5fe-c45a-47df-a307-30e2c6f3fc24 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:49.111417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bc01d5fe-c45a-47df-a307-30e2c6f3fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:49.188878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9dfa0-50f7-4d44-84c7-6502697f6874 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:49.204765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 360/728] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:49.204765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01d5fe-c45a-47df-a307-30e2c6f3fc24 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:49.242483 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:49.243460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.244377 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.245058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.246186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:49.246618 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:49.248337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:49.269669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 369/729] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:49.269669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:49.320721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01d5fe-c45a-47df-a307-30e2c6f3fc24 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:49.362155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878b2bc-40e7-42d7-876e-347721c78d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:49.363294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878b2bc-40e7-42d7-876e-347721c78d4d None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.363817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878b2bc-40e7-42d7-876e-347721c78d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.364315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878b2bc-40e7-42d7-876e-347721c78d4d None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.365374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878b2bc-40e7-42d7-876e-347721c78d4d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:49.367020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878b2bc-40e7-42d7-876e-347721c78d4d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:49.390198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a93957-11c6-48e4-bd80-d087aa190e96 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:49.424442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d3211-5101-445f-81c1-ad368ef183f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:49.425294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d3211-5101-445f-81c1-ad368ef183f6 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.425581 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d3211-5101-445f-81c1-ad368ef183f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.425885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d3211-5101-445f-81c1-ad368ef183f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.437291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d3211-5101-445f-81c1-ad368ef183f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-1c2d3211-5101-445f-81c1-ad368ef183f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:49.445296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:49.446272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 370/730] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:49.461752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d3211-5101-445f-81c1-ad368ef183f6 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:49.564670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d3211-5101-445f-81c1-ad368ef183f6 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:49.594146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 361/731] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:04:49 2022] POST /identity/v3/services => generated 240 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:49.595068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878b2bc-40e7-42d7-876e-347721c78d4d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:49.614797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b424a39-eb9c-423b-9748-138bc7292186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:49.616244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b424a39-eb9c-423b-9748-138bc7292186 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.616817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b424a39-eb9c-423b-9748-138bc7292186 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.617340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b424a39-eb9c-423b-9748-138bc7292186 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.628922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b424a39-eb9c-423b-9748-138bc7292186 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:49.631386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b424a39-eb9c-423b-9748-138bc7292186 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-8b424a39-eb9c-423b-9748-138bc7292186 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:49.652894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:49.653939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:49.654909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:49.657963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 371/732] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:49.667196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562fe7-a6c3-40de-82d8-0468b21cd236 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.667500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562fe7-a6c3-40de-82d8-0468b21cd236 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.667763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562fe7-a6c3-40de-82d8-0468b21cd236 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.669382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 372/733] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:04:49.670138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f562fe7-a6c3-40de-82d8-0468b21cd236 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:49.743928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878b2bc-40e7-42d7-876e-347721c78d4d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:49.786791 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fc578-d863-4e55-bb44-ab6f6546d77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:49.787892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fc578-d863-4e55-bb44-ab6f6546d77b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.788170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fc578-d863-4e55-bb44-ab6f6546d77b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.788444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fc578-d863-4e55-bb44-ab6f6546d77b None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.789497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fc578-d863-4e55-bb44-ab6f6546d77b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:49.789837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fc578-d863-4e55-bb44-ab6f6546d77b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:49.792393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fc578-d863-4e55-bb44-ab6f6546d77b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:49.806267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 362/734] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:49.806267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773751d-ded9-4e11-a0e6-693d2bd0174e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.806267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773751d-ded9-4e11-a0e6-693d2bd0174e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.806267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773751d-ded9-4e11-a0e6-693d2bd0174e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.815562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f562fe7-a6c3-40de-82d8-0468b21cd236 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:49.828318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1773751d-ded9-4e11-a0e6-693d2bd0174e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:49.829500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1773751d-ded9-4e11-a0e6-693d2bd0174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:49.856267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:49.856479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:49.856856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:49.857199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:49.858178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:49.858861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:49.860074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:49.893228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 373/735] 10.222.0.25 () {56 vars in 1135 bytes} [Fri May 13 13:04:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:49.894404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:49.922721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 363/736] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:49.923988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773751d-ded9-4e11-a0e6-693d2bd0174e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.006710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ab2bb-cb89-445a-b983-dce806876791 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.031955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773751d-ded9-4e11-a0e6-693d2bd0174e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.040674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.041634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.042024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.042372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.052686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:50.054299 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:50.068315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d654f72-1c77-4796-97fd-6e898b368e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.068976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d654f72-1c77-4796-97fd-6e898b368e0c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.069273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d654f72-1c77-4796-97fd-6e898b368e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.069586 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d654f72-1c77-4796-97fd-6e898b368e0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:50.073444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:50.074501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:50.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:50.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:50.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:50.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:50.081808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 374/737] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:50.081808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.147285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d654f72-1c77-4796-97fd-6e898b368e0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:50.148676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d654f72-1c77-4796-97fd-6e898b368e0c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:50.192981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb1ff-2b0f-4abc-857f-6621f16cc58c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.258563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 364/738] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:49 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 13 13:04:50.259644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d654f72-1c77-4796-97fd-6e898b368e0c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.355257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac606cb-397d-4382-834c-ceb9b2d9c101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.356211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac606cb-397d-4382-834c-ceb9b2d9c101 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.356587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac606cb-397d-4382-834c-ceb9b2d9c101 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.356953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac606cb-397d-4382-834c-ceb9b2d9c101 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.367052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac606cb-397d-4382-834c-ceb9b2d9c101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:50.369322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d654f72-1c77-4796-97fd-6e898b368e0c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-bac606cb-397d-4382-834c-ceb9b2d9c101 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:50.375026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:50.376129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:50.377152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:50.383395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 375/739] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:50.383395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac606cb-397d-4382-834c-ceb9b2d9c101 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.404448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.405052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.405337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.405620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.406406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:50.406712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:50.408098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:50.424666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 365/740] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:50.425563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.480231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac606cb-397d-4382-834c-ceb9b2d9c101 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.513908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.514700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.514976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.515244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.524974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:50.526464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:50.533987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c0c57-3764-46cc-b454-4a0a4a3cf4de None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:50.545047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:50.545983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:50.546872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:50.546872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:50.546872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:50.546872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:50.546872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 376/741] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:50.547157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.572375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc720-e8f0-4bf2-9be8-5caf252e3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.573313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc720-e8f0-4bf2-9be8-5caf252e3b92 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.573595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc720-e8f0-4bf2-9be8-5caf252e3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.573896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc720-e8f0-4bf2-9be8-5caf252e3b92 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.574632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc720-e8f0-4bf2-9be8-5caf252e3b92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:50.574933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc720-e8f0-4bf2-9be8-5caf252e3b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:50.576298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc720-e8f0-4bf2-9be8-5caf252e3b92 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:50.588464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 366/742] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:50.589741 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a49cc-dac5-4945-a4cf-950244a3c93b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.590126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a49cc-dac5-4945-a4cf-950244a3c93b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.590408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a49cc-dac5-4945-a4cf-950244a3c93b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.593961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 367/743] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:50.593961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a49cc-dac5-4945-a4cf-950244a3c93b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.656196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765d4e7-2f01-44eb-830c-ce97b4c5ec07 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.690061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.690888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.691174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.691454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.700082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:50.701564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:50.717149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:50.718551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:50.719570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:50.722156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 377/744] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:50.723724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.819186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a49cc-dac5-4945-a4cf-950244a3c93b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.824560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4282ec-502e-4d77-bcb1-26ebad1a5d7d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:50.854206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dacaa-dcab-492e-82ff-062a8448fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.854897 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dacaa-dcab-492e-82ff-062a8448fa94 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.855183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dacaa-dcab-492e-82ff-062a8448fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.855465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dacaa-dcab-492e-82ff-062a8448fa94 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.856206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dacaa-dcab-492e-82ff-062a8448fa94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:50.856516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dacaa-dcab-492e-82ff-062a8448fa94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:50.857960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dacaa-dcab-492e-82ff-062a8448fa94 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:50.866315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:50.867696 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.868099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.868382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.874485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 368/745] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:50.874485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2d4d1-cd95-43cc-a5bf-c493b05cc935 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:50.874485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2d4d1-cd95-43cc-a5bf-c493b05cc935 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:50.874485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2d4d1-cd95-43cc-a5bf-c493b05cc935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:50.884724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:50.886230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:50.887647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-08f2d4d1-cd95-43cc-a5bf-c493b05cc935 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:50.889074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-08f2d4d1-cd95-43cc-a5bf-c493b05cc935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:50.903449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:50.904468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:50.905533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:50.905533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:50.905533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:50.905533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:50.905533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 378/746] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:50.905533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:50.959503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 369/747] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:50 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 13 13:04:50.960686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2d4d1-cd95-43cc-a5bf-c493b05cc935 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.024577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a77251-29c5-41c7-a6ec-de4020e3ef3f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.055965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2d4d1-cd95-43cc-a5bf-c493b05cc935 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.058593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.059268 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.059599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.059867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.060744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:51.061052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.062485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.079712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 379/748] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:51.080587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.093149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.094113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.094396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.094671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.101222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.102688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:51.116637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:51.117642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:51.118600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 370/749] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:51.119147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.180045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf467e1-be74-4f1a-8800-34fc7830fc50 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.214925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.215787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.216215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.216329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7aeb4-b738-49ca-8bd0-b2fc9d38e16e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.216734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.217759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:51.218929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.220441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.233960 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 380/750] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:51.234892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.248238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996669f-52ed-4e80-bf29-455f2c46412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.249015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996669f-52ed-4e80-bf29-455f2c46412a None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.249404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996669f-52ed-4e80-bf29-455f2c46412a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.249779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996669f-52ed-4e80-bf29-455f2c46412a None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.250635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996669f-52ed-4e80-bf29-455f2c46412a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.252098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996669f-52ed-4e80-bf29-455f2c46412a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.333818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 371/751] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:04:51 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:51.333818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996669f-52ed-4e80-bf29-455f2c46412a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.334237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43a95-4b19-4a62-9731-3e587868db1e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.376663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.377467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.377756 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.378557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.393225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.395227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:51.419846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:51.420848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:51.421758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:51.421758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:51.421758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:51.421758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:51.421758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 381/752] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:51.428217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.450404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996669f-52ed-4e80-bf29-455f2c46412a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.502694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.503750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.504239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.504730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.505738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:51.506277 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.507936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.520972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 372/753] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:51.522276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.546682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea5f65-bb73-403f-a25b-e3527c36afb5 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.589375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.590255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.590606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.590884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.601862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.603441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:51.619371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:51.620470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:51.621400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 382/754] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:04:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:51.622026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.654227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1359e-5cf9-42ce-89a4-cff949a148c5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.705238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.706329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.706809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.707278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.708553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:51.709063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.711513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.730134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 373/755] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:51.731469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.781385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bb3a7-5fb6-404b-9a40-f65d23e72e45 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.830443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.831277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.831998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.832627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.833919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=96f6d41c529444fba00bb09ac4224eaa) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:51.834599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.836321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.845071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 383/756] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:51 2022] GET /identity/v3/role_assignments?user.id=96f6d41c529444fba00bb09ac4224eaa&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=663341777b0b44128ed78fe8d40f6462 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:51.846602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:51.860151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9738b37-dbec-4871-afce-9bae5282c23f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:51.966684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:51.967585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:51.967980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:51.968365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:51.969340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:51.969763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:51.971414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:51.998221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 374/757] 10.222.0.25 () {56 vars in 1149 bytes} [Fri May 13 13:04:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:52.004435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:52.108485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21106931-09e0-487b-b0ad-9d997ebbddcc None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:52.134951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc93db2-b51e-4ffe-8b61-42a065f57a8c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:52.149980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f317-3cd9-4063-9dbb-61fc303fc79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:52.150936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f317-3cd9-4063-9dbb-61fc303fc79f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.151329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f317-3cd9-4063-9dbb-61fc303fc79f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.151718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f317-3cd9-4063-9dbb-61fc303fc79f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.158565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f317-3cd9-4063-9dbb-61fc303fc79f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:52.160144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f317-3cd9-4063-9dbb-61fc303fc79f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:52.176209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044476-a487-4299-9880-22bdcf3d2d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:52.177619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044476-a487-4299-9880-22bdcf3d2d06 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.178354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044476-a487-4299-9880-22bdcf3d2d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.179052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044476-a487-4299-9880-22bdcf3d2d06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-0a62f317-3cd9-4063-9dbb-61fc303fc79f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:52.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:52.183434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:52.185444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:52.185444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:52.185444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:52.185444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:52.185444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:52.185444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:52.185444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:52.185444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:52.190388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 384/758] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:52.192181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4446-9763-45c9-afb2-b5b30b89eb46 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.192181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4446-9763-45c9-afb2-b5b30b89eb46 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.192650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4446-9763-45c9-afb2-b5b30b89eb46 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.194351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 385/759] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:52.199161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4446-9763-45c9-afb2-b5b30b89eb46 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:52.282714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0044476-a487-4299-9880-22bdcf3d2d06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:52.285053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0044476-a487-4299-9880-22bdcf3d2d06 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:52.301719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4446-9763-45c9-afb2-b5b30b89eb46 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:52.337076 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227e7dd-2a52-4675-be4f-40838e3fe7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:52.337982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227e7dd-2a52-4675-be4f-40838e3fe7ac None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.338417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227e7dd-2a52-4675-be4f-40838e3fe7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.338817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227e7dd-2a52-4675-be4f-40838e3fe7ac None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.339739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227e7dd-2a52-4675-be4f-40838e3fe7ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:52.340164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227e7dd-2a52-4675-be4f-40838e3fe7ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:52.341938 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227e7dd-2a52-4675-be4f-40838e3fe7ac None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:52.357868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 386/760] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:52.359619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46cfaa-85d7-417a-a048-6e2a0e07bd48 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.359741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46cfaa-85d7-417a-a048-6e2a0e07bd48 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.360142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46cfaa-85d7-417a-a048-6e2a0e07bd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.378445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2f46cfaa-85d7-417a-a048-6e2a0e07bd48 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:52.381181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2f46cfaa-85d7-417a-a048-6e2a0e07bd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:52.393447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 375/761] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:04:52.394549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044476-a487-4299-9880-22bdcf3d2d06 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:52.463189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 387/762] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:04:52.463189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46cfaa-85d7-417a-a048-6e2a0e07bd48 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:52.527949 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044476-a487-4299-9880-22bdcf3d2d06 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:52.571525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20ce11-3ead-448e-8d8e-55ba41842aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:52.572825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20ce11-3ead-448e-8d8e-55ba41842aa4 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.573717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20ce11-3ead-448e-8d8e-55ba41842aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.574253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20ce11-3ead-448e-8d8e-55ba41842aa4 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/3f94e52e32324d8cbf38fb05dc38ea96/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.577167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20ce11-3ead-448e-8d8e-55ba41842aa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=3f94e52e32324d8cbf38fb05dc38ea96, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:52.579252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20ce11-3ead-448e-8d8e-55ba41842aa4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:52.585067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46cfaa-85d7-417a-a048-6e2a0e07bd48 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:52.618083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:52.619447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.619723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.619994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.635232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:52.636720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:52.651867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 376/763] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:04:52 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/3f94e52e32324d8cbf38fb05dc38ea96/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:04:52.653164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20ce11-3ead-448e-8d8e-55ba41842aa4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:52.656662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:52.657587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:52.658530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:52.658530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:52.658530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:52.658530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:52.658530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 388/764] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:52.664800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:52.818487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20ce11-3ead-448e-8d8e-55ba41842aa4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:52.828547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efc73c-0acf-47a5-9030-a82cdc08d8b3 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:52.871894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:52.872987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.874341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.874831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.875860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:52.876780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:52.879831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:52.891533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:52.892412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:52.892700 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:52.892988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] PATH_INFO: `/v3/services/network` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:52.893984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:52.895365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:52.896018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 389/765] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:52.901220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:52.908653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:52.909867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:52.910923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:52.918429 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 377/766] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:52 2022] GET /identity/v3/services/network => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:52.924276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:53.027593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d0ff8-5a5d-4639-9748-a747b4e50829 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:53.048169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e316ac-2759-485d-8d29-7637c0b16982 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:53.062634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:53.064349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:53.064964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:53.066744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:53.079988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:53.082611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:53.110409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:53.111394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:53.111841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:53.112298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:53.113238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] RBAC: Adding query filter params (name=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:53.113698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:53.113892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:53.114912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:53.115995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:53.115995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:53.115995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:53.115995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:53.115995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:53.115995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:53.115995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:53.115995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:53.118004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:53.118120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 390/767] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:53.123369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:53.123917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 378/768] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:04:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:53.136586 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:53.240289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5705c-7fc8-4484-bfc2-b116e7b3f986 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:53.284314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:53.285140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:53.285528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:53.285937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:53.286836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:53.287250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:53.288786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:53.303261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 391/769] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:53.310563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:53.368100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bef30-0f81-4336-ac63-9cfcc5b302ce None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:53.423055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564088da-71c6-4e32-a0e0-42f6f9f3ba24 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:53.447117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:53.447798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:53.448085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:53.448366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:53.449085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] RBAC: Adding query filter params (type=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:53.449394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:53.450801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:53.459848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 379/770] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:04:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:53.465324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:53.466025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:53.466310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:53.466587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:53.467411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=63d11bc6b12949f28957146b7d66a71d) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:53.467722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:53.469150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:53.470728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:53.488891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 392/771] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:53 2022] GET /identity/v3/role_assignments?user.id=63d11bc6b12949f28957146b7d66a71d&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:53.642169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9ef10-c678-4111-9205-831f5084c6be None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:53.695719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7e13a-00d1-440d-afdc-cd8578e87877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:53.696407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e13a-00d1-440d-afdc-cd8578e87877 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:53.696728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e13a-00d1-440d-afdc-cd8578e87877 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:53.697018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e13a-00d1-440d-afdc-cd8578e87877 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:53.697853 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7e13a-00d1-440d-afdc-cd8578e87877 None admin] RBAC: Adding query filter params (interface=public, service_id=b63c2075806d4ce8a4e94f1478a3a1f0, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:53.698169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7e13a-00d1-440d-afdc-cd8578e87877 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:53.699564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7e13a-00d1-440d-afdc-cd8578e87877 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:53.714381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 380/772] 10.222.0.25 () {56 vars in 1251 bytes} [Fri May 13 13:04:53 2022] GET /identity/v3/endpoints?service_id=b63c2075806d4ce8a4e94f1478a3a1f0&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:53.719831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:53.834472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e2f72-c998-4f87-bed4-d209e6487111 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:53.874898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffd9f7-a6bc-461c-beab-076d7e807e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:53.875572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffd9f7-a6bc-461c-beab-076d7e807e74 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:53.875851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffd9f7-a6bc-461c-beab-076d7e807e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:53.876124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffd9f7-a6bc-461c-beab-076d7e807e74 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:53.876804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ffd9f7-a6bc-461c-beab-076d7e807e74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:53.878185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ffd9f7-a6bc-461c-beab-076d7e807e74 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:53.886919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 393/773] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:04:53 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:54.139167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fbf05-2b83-4878-91fe-86fe70de2638 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:54.139868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fbf05-2b83-4878-91fe-86fe70de2638 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:54.139868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fbf05-2b83-4878-91fe-86fe70de2638 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:54.141353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 381/774] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:54.146289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ca320-dd9b-47b4-b895-36d07f788fe4 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:54.146581 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ca320-dd9b-47b4-b895-36d07f788fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:54.146859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ca320-dd9b-47b4-b895-36d07f788fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:54.163524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-506ca320-dd9b-47b4-b895-36d07f788fe4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:54.164706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-506ca320-dd9b-47b4-b895-36d07f788fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:54.247488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 394/775] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:04:54.269639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fbf05-2b83-4878-91fe-86fe70de2638 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:54.371701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fbf05-2b83-4878-91fe-86fe70de2638 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:54.387738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b15e6-4adb-4cee-87fe-f1d083ac76ca None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:54.388032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b15e6-4adb-4cee-87fe-f1d083ac76ca None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:54.388303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b15e6-4adb-4cee-87fe-f1d083ac76ca None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:54.393574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 395/776] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:04:54.394712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b735d9-9b7c-45d8-89cb-f2eeaf8d9263 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:54.395004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b735d9-9b7c-45d8-89cb-f2eeaf8d9263 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:54.395289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b735d9-9b7c-45d8-89cb-f2eeaf8d9263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:54.407880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:54.408959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:54.409248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:54.409522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:54.410211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:54.411575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:54.412497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e7b735d9-9b7c-45d8-89cb-f2eeaf8d9263 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:54.413692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e7b735d9-9b7c-45d8-89cb-f2eeaf8d9263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:54.426306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:54.427828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:54.429440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 382/777] 10.222.0.25 () {56 vars in 1129 bytes} [Fri May 13 13:04:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:54.434785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:54.548041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 396/778] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:04:54.566326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfa785-1ac0-452f-9e2a-8173d9d16ed6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:54.612662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b735d9-9b7c-45d8-89cb-f2eeaf8d9263 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:54.615125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:54.615774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:54.616066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:54.616347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:54.621876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:54.621876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:54.621876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:54.624494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 383/779] 10.222.0.25 () {56 vars in 1124 bytes} [Fri May 13 13:04:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:54.635870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:54.752998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a100c6-a7a5-4db5-ba36-a8d0cace89a5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:54.799031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b735d9-9b7c-45d8-89cb-f2eeaf8d9263 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:54.826584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:54.828613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:54.829052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:54.830230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:54.832345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:54.832345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:54.835819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:54.844144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 384/780] 10.222.0.25 () {56 vars in 1124 bytes} [Fri May 13 13:04:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:54.856987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:54.858834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589c7ea-369d-4445-9550-55eaed4a80f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:54.859642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589c7ea-369d-4445-9550-55eaed4a80f9 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:54.859921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589c7ea-369d-4445-9550-55eaed4a80f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:54.860205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589c7ea-369d-4445-9550-55eaed4a80f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:54.871181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589c7ea-369d-4445-9550-55eaed4a80f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-7589c7ea-369d-4445-9550-55eaed4a80f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:54.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:54.879941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:54.881006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 397/781] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:54.890769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589c7ea-369d-4445-9550-55eaed4a80f9 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:54.967702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd33b0-3455-4e82-adb4-cbd823a2f3c2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:55.010777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:55.011753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.012222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.012681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.013650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] RBAC: Adding query filter params (interface=public, service_id=154d453c2fa24bb88669540372aa63c6, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:55.014173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:55.015743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:55.021424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 385/782] 10.222.0.25 () {56 vars in 1251 bytes} [Fri May 13 13:04:54 2022] GET /identity/v3/endpoints?service_id=154d453c2fa24bb88669540372aa63c6&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:55.027275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589c7ea-369d-4445-9550-55eaed4a80f9 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:55.034594 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:55.068193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:55.068791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.069082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.069357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.070134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:55.070439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:55.072019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:55.080762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 398/783] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:55.091037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:55.167317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccb7e0-b70d-471c-97ee-5418f98131e6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:55.189518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaab44-f146-43dc-9feb-bb9e35205574 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:55.229413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69905c60-f39b-4c03-91f3-afce80495c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:55.230110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69905c60-f39b-4c03-91f3-afce80495c68 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.230392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69905c60-f39b-4c03-91f3-afce80495c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.230668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69905c60-f39b-4c03-91f3-afce80495c68 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.231350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69905c60-f39b-4c03-91f3-afce80495c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:55.232705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69905c60-f39b-4c03-91f3-afce80495c68 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:55.236068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:55.236867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.237143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.237411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.248806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 386/784] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:04:55 2022] GET /identity/v3/services => generated 772 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:55.252486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:55.254082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:55.272936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:55.273889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:55.275325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:55.275325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:55.275325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:55.275325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:55.275325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 399/785] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:55.283294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69905c60-f39b-4c03-91f3-afce80495c68 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:55.433377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69905c60-f39b-4c03-91f3-afce80495c68 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:55.485134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed0a31-67e2-4ab5-8f33-ae42f4f76033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:55.485843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0a31-67e2-4ab5-8f33-ae42f4f76033 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.486136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0a31-67e2-4ab5-8f33-ae42f4f76033 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.486418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0a31-67e2-4ab5-8f33-ae42f4f76033 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.487167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed0a31-67e2-4ab5-8f33-ae42f4f76033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:55.487478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed0a31-67e2-4ab5-8f33-ae42f4f76033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:55.488885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed0a31-67e2-4ab5-8f33-ae42f4f76033 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:55.511982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 387/786] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:55.517571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:55.532892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f00c2-4d8e-4c99-b833-5f2f2519a9e8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.533483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f00c2-4d8e-4c99-b833-5f2f2519a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.533766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f00c2-4d8e-4c99-b833-5f2f2519a9e8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.535303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 388/787] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:04:55.541906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafefb12-b2e2-4ef5-9b69-bca3f3f17b28 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.542201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafefb12-b2e2-4ef5-9b69-bca3f3f17b28 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.542488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafefb12-b2e2-4ef5-9b69-bca3f3f17b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.573257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-eafefb12-b2e2-4ef5-9b69-bca3f3f17b28 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:55.575355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-eafefb12-b2e2-4ef5-9b69-bca3f3f17b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:55.668273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39962d18-9af9-4d22-bc76-2ae38babe8d9 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:55.673902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 389/788] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:04:55.700613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafefb12-b2e2-4ef5-9b69-bca3f3f17b28 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:55.716821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:55.717882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.718173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.718448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.731609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:55.733506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:55.748575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:55.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:04:55.750647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:55.753269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 400/789] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:55.758993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:55.862408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be946468-ce5e-40d1-9239-2942871fc55f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:55.885501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafefb12-b2e2-4ef5-9b69-bca3f3f17b28 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:55.905007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:55.905702 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.906014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.906294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.907148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:55.907465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:55.908904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:55.934738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26b9f0-71cd-4459-897b-d40ae6f51c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:55.936173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26b9f0-71cd-4459-897b-d40ae6f51c4d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:55.936664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26b9f0-71cd-4459-897b-d40ae6f51c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:55.937142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26b9f0-71cd-4459-897b-d40ae6f51c4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:55.937590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 401/790] 10.222.0.25 () {56 vars in 1135 bytes} [Fri May 13 13:04:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:55.942785 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:55.949614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26b9f0-71cd-4459-897b-d40ae6f51c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0a26b9f0-71cd-4459-897b-d40ae6f51c4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:55.958685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:55.959622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:55.960450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:55.969118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 390/791] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:55.972072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26b9f0-71cd-4459-897b-d40ae6f51c4d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:56.114459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26b9f0-71cd-4459-897b-d40ae6f51c4d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.143502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29d31-678f-468e-b182-fa0488896138 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.166740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:56.167457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.167996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.167996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.168734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:56.169037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:56.170864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:56.181458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 391/792] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:04:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:56.187347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:56.202989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:56.204196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.204728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.205204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.234268 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:56.243419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:56.277021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:56.278882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:56.280687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:56.280687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:56.280687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:56.280687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:56.280687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 402/793] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:04:56.286279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:56.386223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac89f4-3d43-4859-81af-73f234c5dc02 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.410421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978146c-3a20-4d48-9089-bd365e9ec598 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.426566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb88e5-c795-473f-acba-8b8a5a401b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:56.427387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb88e5-c795-473f-acba-8b8a5a401b20 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.427676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb88e5-c795-473f-acba-8b8a5a401b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.427958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb88e5-c795-473f-acba-8b8a5a401b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.443905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb88e5-c795-473f-acba-8b8a5a401b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:56.445487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb88e5-c795-473f-acba-8b8a5a401b20 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:56.461658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d74dab-650f-43b6-b6d5-3d8b7f63ccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:56.462952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d74dab-650f-43b6-b6d5-3d8b7f63ccab None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.463474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d74dab-650f-43b6-b6d5-3d8b7f63ccab None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-d7cb88e5-c795-473f-acba-8b8a5a401b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:56.463963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:56.464856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:56.465753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:56.465753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:56.465753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:56.465753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:56.467097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d74dab-650f-43b6-b6d5-3d8b7f63ccab None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.468207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d74dab-650f-43b6-b6d5-3d8b7f63ccab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:56.468731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d74dab-650f-43b6-b6d5-3d8b7f63ccab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:56.470001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 392/794] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:56.470997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d74dab-650f-43b6-b6d5-3d8b7f63ccab None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:56.471387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2772e-1e24-48e6-a902-01312d60f8a5 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.471735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2772e-1e24-48e6-a902-01312d60f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.471965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2772e-1e24-48e6-a902-01312d60f8a5 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.475816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 393/795] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:56.475816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2772e-1e24-48e6-a902-01312d60f8a5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:56.500447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 403/796] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:56.503093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84344978-e3ed-4e82-8d34-24142ed3ccae None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.503979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84344978-e3ed-4e82-8d34-24142ed3ccae None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.504546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84344978-e3ed-4e82-8d34-24142ed3ccae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.527609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-84344978-e3ed-4e82-8d34-24142ed3ccae None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:56.529212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-84344978-e3ed-4e82-8d34-24142ed3ccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:56.606090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 404/797] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:04:56.607022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84344978-e3ed-4e82-8d34-24142ed3ccae None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:56.608349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2772e-1e24-48e6-a902-01312d60f8a5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.650674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:56.651856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.652683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.653169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.654391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:56.654908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:56.657407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:56.676295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 394/798] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:56.677555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:56.733669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84344978-e3ed-4e82-8d34-24142ed3ccae None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.767409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:56.768211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.768515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.768786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.775875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:56.777333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:56.791009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:56.792035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:56.792927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 405/799] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:56.793879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:56.794243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb6459-bc84-4641-9bea-ecb5cf0e0afe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.833227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b5665-bec3-4922-a5c3-09dfc129a816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:56.834157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5665-bec3-4922-a5c3-09dfc129a816 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.834437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5665-bec3-4922-a5c3-09dfc129a816 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.834713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5665-bec3-4922-a5c3-09dfc129a816 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.845170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b5665-bec3-4922-a5c3-09dfc129a816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c51b5665-bec3-4922-a5c3-09dfc129a816 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:56.853104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:56.854342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:56.855674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 395/800] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:04:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:56.856386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5726e6-a4a1-4363-a2dd-f3b96a3c072f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.856386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5726e6-a4a1-4363-a2dd-f3b96a3c072f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.856510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5726e6-a4a1-4363-a2dd-f3b96a3c072f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.862097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 396/801] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:56.862097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5726e6-a4a1-4363-a2dd-f3b96a3c072f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:56.914859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d90e4-ef1b-4046-9d77-263d2da4d2c7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.953361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:56.954186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:56.954475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:56.954755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:56.963101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:56.964563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:56.968441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5726e6-a4a1-4363-a2dd-f3b96a3c072f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:56.979088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:56.980263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 406/802] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:04:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:56.981424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.011246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913e366-6bcc-4e7e-bc6e-193011f0eae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.013065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913e366-6bcc-4e7e-bc6e-193011f0eae2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.013360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913e366-6bcc-4e7e-bc6e-193011f0eae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.013643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913e366-6bcc-4e7e-bc6e-193011f0eae2 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.014466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913e366-6bcc-4e7e-bc6e-193011f0eae2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:57.014783 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913e366-6bcc-4e7e-bc6e-193011f0eae2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.016373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913e366-6bcc-4e7e-bc6e-193011f0eae2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.028778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 397/803] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:57.033724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9397b-c1cf-4cae-ae5e-ea067d6aa7a9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.033724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9397b-c1cf-4cae-ae5e-ea067d6aa7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.033724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9397b-c1cf-4cae-ae5e-ea067d6aa7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.046895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-10f9397b-c1cf-4cae-ae5e-ea067d6aa7a9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:57.048097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-10f9397b-c1cf-4cae-ae5e-ea067d6aa7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:57.100452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27167e28-3ccd-40ed-8bf2-b3d8ae089cf3 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:57.123712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 398/804] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:04:57.124864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9397b-c1cf-4cae-ae5e-ea067d6aa7a9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.149093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.149826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.150120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.150402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.151158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:57.151533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.152913 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.163626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 407/805] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:04:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:57.166325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.239920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9397b-c1cf-4cae-ae5e-ea067d6aa7a9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:57.275840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.276765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.277148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.277530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.278535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:57.278914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.280418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.280734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326df8a-44b0-4260-b791-3a12d205e11f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:57.298394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 399/806] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:04:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:57.299411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.320822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630fe33-da3c-4759-995c-c69979dfc302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.321746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630fe33-da3c-4759-995c-c69979dfc302 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.322059 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630fe33-da3c-4759-995c-c69979dfc302 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.322339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630fe33-da3c-4759-995c-c69979dfc302 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/96f6d41c529444fba00bb09ac4224eaa/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.324308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630fe33-da3c-4759-995c-c69979dfc302 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=96f6d41c529444fba00bb09ac4224eaa, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.325743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630fe33-da3c-4759-995c-c69979dfc302 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.377384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 408/807] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:04:57 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/96f6d41c529444fba00bb09ac4224eaa/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:04:57.378466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630fe33-da3c-4759-995c-c69979dfc302 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.413724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9041f-06b2-4038-9be8-6aef5d352076 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:57.459503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.460579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.461117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.461603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.463033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.464660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:57.477697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:57.479351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 13 13:04:57.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:57.488699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 400/808] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:57 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:57.488837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630fe33-da3c-4759-995c-c69979dfc302 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:57.490201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.527211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.527866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.528148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.528421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.540485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.542089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:57.561129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:57.562160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:57.563054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:57.563054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:57.563054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:57.563054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:57.563054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 409/809] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:57.563789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.612381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59058f6-e272-45a3-9851-374e673102e5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:57.654691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151375-4db7-4f67-b698-0c0b8e42eeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.655340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151375-4db7-4f67-b698-0c0b8e42eeab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.655773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151375-4db7-4f67-b698-0c0b8e42eeab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.656086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151375-4db7-4f67-b698-0c0b8e42eeab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.674052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e9b3f-78af-4366-a277-b144ab934072 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:57.714442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.715126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.715488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.715838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.716619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] RBAC: Adding query filter params (name=network) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:57.717000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.718497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.723518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 410/810] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:04:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:57.728782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.741018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c151375-4db7-4f67-b698-0c0b8e42eeab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.743003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c151375-4db7-4f67-b698-0c0b8e42eeab None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.847369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cc30c-59b7-4e98-825e-078700a236da None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:57.873888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 401/811] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:04:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:04:57.874780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151375-4db7-4f67-b698-0c0b8e42eeab None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.887552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:57.888503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:57.888784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:57.889060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:57.889980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:57.890288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:57.892295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:57.907573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 411/812] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:57.908411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:57.985287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151375-4db7-4f67-b698-0c0b8e42eeab None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.015882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15395abd-0cb2-4c37-a3e3-05a909a46d3a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.026403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.027197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.027587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.028457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.029275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] RBAC: Adding query filter params (type=network) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:58.029683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.031153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.040369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 402/813] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:04:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:58.041272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.049468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa0bbe-e149-4219-b361-4d5678c6bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.050292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0bbe-e149-4219-b361-4d5678c6bad2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.050583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0bbe-e149-4219-b361-4d5678c6bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.050860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0bbe-e149-4219-b361-4d5678c6bad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.061211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa0bbe-e149-4219-b361-4d5678c6bad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.062774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa0bbe-e149-4219-b361-4d5678c6bad2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-81fa0bbe-e149-4219-b361-4d5678c6bad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:58.079887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:58.080948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:58.082051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:58.082051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:58.082051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:58.082051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:58.085642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 412/814] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:58.086757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a454c-063a-42ca-b074-3c9a704bb7cb None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.087047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a454c-063a-42ca-b074-3c9a704bb7cb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.087887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a454c-063a-42ca-b074-3c9a704bb7cb None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.089428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 413/815] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:58.090153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a454c-063a-42ca-b074-3c9a704bb7cb None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.143655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7a42-511a-460a-83ac-3d3a35bd1a4e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.176092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.176911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.177197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.177501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.185523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.187200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.194237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a454c-063a-42ca-b074-3c9a704bb7cb None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:04:58.206023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:58.207071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:04:58.208221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:58.214673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 403/816] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:58.215654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.229173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47380de-b500-481b-98cc-9380462b840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.229861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47380de-b500-481b-98cc-9380462b840f None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.230150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47380de-b500-481b-98cc-9380462b840f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.230427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47380de-b500-481b-98cc-9380462b840f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.231163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47380de-b500-481b-98cc-9380462b840f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.232532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47380de-b500-481b-98cc-9380462b840f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.316925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdf1c4-9cf5-4b3d-a14e-590be4c64253 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.337073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 414/817] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:04:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:58.338534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420abbfa-d24a-446e-b541-0c4723059528 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.338886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420abbfa-d24a-446e-b541-0c4723059528 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.341404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420abbfa-d24a-446e-b541-0c4723059528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.357007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-420abbfa-d24a-446e-b541-0c4723059528 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:58.358553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-420abbfa-d24a-446e-b541-0c4723059528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:58.371693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.372647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.375915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.375915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.375915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:58.375915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.376558 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.390839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 404/818] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:58.392034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.442232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 415/819] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:04:58.443348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420abbfa-d24a-446e-b541-0c4723059528 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.501442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b517ad-ec63-42ec-99e5-d8f2c2f86428 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.542828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.543774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.544317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.544732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.545842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:58.546312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.547930 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.569125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 405/820] 10.222.0.25 () {56 vars in 1149 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:58.575264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.581389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420abbfa-d24a-446e-b541-0c4723059528 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.624450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.625611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.625933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.626213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.634120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.635630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:58.650718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:58.651636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:58.652515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 416/821] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:58.653017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.687385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063b5bc-1cae-48f2-83d5-8950f919537a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.719683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.720487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.720765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.721035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.721681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.723081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:58.738265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:58.739225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:04:58.740069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 406/822] 10.222.0.25 () {56 vars in 1129 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:58.740918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.758809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05b4cd-3249-4f9e-b8bb-3f8ad6b34eb0 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.792420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.793209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.793492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.793763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.803883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.805326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:04:58.822758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:58.823661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:58.824586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:04:58.824586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:04:58.824586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:04:58.824586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:58.824586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 417/823] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:58.824586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.844064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a07425-58ec-483e-aa69-c8e7b09b3578 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.880486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.881772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.882397 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.882885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.884252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:58.884648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.886521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.899244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 407/824] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:58.900837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:58.928620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe0eaa-6970-45c4-96ae-551166719d86 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:58.961203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:58.961895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:58.962201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:58.962972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:58.963671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:58.963976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:58.965340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:58.969666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 418/825] 10.222.0.25 () {56 vars in 1124 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:58.970463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:59.010501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8290998-83e0-4e15-9474-b53ffb8dbb7b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:59.044107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:59.044763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.045046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.045326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.046072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:59.046386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:59.047732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:59.059030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 408/826] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:59.059657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:59.073550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c5e4-3de5-4692-abee-39fe4606fb96 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:59.107430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:59.108091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.108377 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.108663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.109488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=3f94e52e32324d8cbf38fb05dc38ea96) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:59.109824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:59.111266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:59.116992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 419/827] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:04:58 2022] GET /identity/v3/role_assignments?user.id=3f94e52e32324d8cbf38fb05dc38ea96&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=663341777b0b44128ed78fe8d40f6462 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:59.117766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:59.176844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b637e13-e65c-4210-a6bd-cedc1157b60e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:59.222996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:59.224592 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.225084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.225513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.226848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:59.227323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:59.229622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:59.242035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 409/828] 10.222.0.25 () {56 vars in 1124 bytes} [Fri May 13 13:04:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:04:59.244078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:59.260259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03e598-16e8-4333-a822-35e0b20f085a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:59.305168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:59.306483 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.306921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.307688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.315953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:59.317609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:04:59.335363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:59.336361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:59.337910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 420/829] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:04:59.342607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:59.362984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66846317-06df-44de-b9a7-85497df1d6fb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:59.406078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a805d-fa8a-4bd0-a338-885eda34d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:59.407266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a805d-fa8a-4bd0-a338-885eda34d011 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.407754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a805d-fa8a-4bd0-a338-885eda34d011 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.408333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a805d-fa8a-4bd0-a338-885eda34d011 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.409536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a805d-fa8a-4bd0-a338-885eda34d011 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:59.411473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a805d-fa8a-4bd0-a338-885eda34d011 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:59.446640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187f995-c304-47e7-9ef8-6d7870612631 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:59.482411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:59.483105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.483510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.483797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.484611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:04:59.484922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:59.486447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:59.502870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 421/830] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:04:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:04:59.508855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:59.517017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 410/831] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:04:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:04:59.628395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6199a-5dc8-4e0a-82a7-c0454598ad0f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:59.671331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532f125-59d0-4d57-a995-dbcd2885a7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:59.672233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f125-59d0-4d57-a995-dbcd2885a7a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.672516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f125-59d0-4d57-a995-dbcd2885a7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.672789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f125-59d0-4d57-a995-dbcd2885a7a7 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/63d11bc6b12949f28957146b7d66a71d/roles/3445c3dc3ab4448aaa19f3d511a884b8` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.683406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc608dc3-a5ff-4a2c-892b-8b4cccbc07aa None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.683747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc608dc3-a5ff-4a2c-892b-8b4cccbc07aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.683968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc608dc3-a5ff-4a2c-892b-8b4cccbc07aa None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.689614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 411/832] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:04:59.689816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532f125-59d0-4d57-a995-dbcd2885a7a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=63d11bc6b12949f28957146b7d66a71d, role_id=3445c3dc3ab4448aaa19f3d511a884b8)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:04:59.691213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532f125-59d0-4d57-a995-dbcd2885a7a7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:04:59.691498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea2a9f-d62f-494f-b667-fae32b031a07 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.691791 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea2a9f-d62f-494f-b667-fae32b031a07 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.692068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea2a9f-d62f-494f-b667-fae32b031a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.710535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-daea2a9f-d62f-494f-b667-fae32b031a07 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:04:59.711688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-daea2a9f-d62f-494f-b667-fae32b031a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:04:59.776971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 422/833] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:04:59 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/63d11bc6b12949f28957146b7d66a71d/roles/3445c3dc3ab4448aaa19f3d511a884b8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:04:59.784872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 412/834] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:04:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:04:59.817241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532f125-59d0-4d57-a995-dbcd2885a7a7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:04:59.934934 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532f125-59d0-4d57-a995-dbcd2885a7a7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:04:59.973285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0dfed-2aab-49ee-baa8-54a32d96b441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:04:59.974396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0dfed-2aab-49ee-baa8-54a32d96b441 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:04:59.974829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0dfed-2aab-49ee-baa8-54a32d96b441 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:04:59.975248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0dfed-2aab-49ee-baa8-54a32d96b441 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:04:59.986069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0dfed-2aab-49ee-baa8-54a32d96b441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-77f0dfed-2aab-49ee-baa8-54a32d96b441 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:04:59.994858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:04:59.996015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:04:59.997000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:04:59.999285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 423/835] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:04:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:00.005203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea2a9f-d62f-494f-b667-fae32b031a07 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:00.110338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea2a9f-d62f-494f-b667-fae32b031a07 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:00.144339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:00.145281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:00.145711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:00.146156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:00.147043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:00.147527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:00.149063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:00.162187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 413/836] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:00.166708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0dfed-2aab-49ee-baa8-54a32d96b441 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:00.271063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0dfed-2aab-49ee-baa8-54a32d96b441 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:00.321580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b854680-d90f-428c-aa65-9242693379de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:00.322921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854680-d90f-428c-aa65-9242693379de None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:00.322921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854680-d90f-428c-aa65-9242693379de None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:00.322921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854680-d90f-428c-aa65-9242693379de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:00.334500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b854680-d90f-428c-aa65-9242693379de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:00.336145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b854680-d90f-428c-aa65-9242693379de None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-5b854680-d90f-428c-aa65-9242693379de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:00.354737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:00.355855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:00.356939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:00.356939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:00.356939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:00.356939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:00.358532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 424/837] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:00.363833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:00.468591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913834f-b69d-4227-b607-f337a60ea22a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:00.502494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:00.503161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:00.503444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:00.503719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:00.504444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:00.504749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:00.506155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:00.518285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 414/838] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:00.523640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b854680-d90f-428c-aa65-9242693379de None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:00.624439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b854680-d90f-428c-aa65-9242693379de None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:00.658616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:00.659767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:00.660276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:00.660779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:00.668722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:00.670439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:00.685491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:00.686739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 13 13:05:00.687698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:00.690652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 425/839] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:05:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:00.696705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:00.798030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2262aa3-5874-4d91-8d34-0d23e52ad34d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:00.835234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:00.836269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:00.836815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:00.837357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:00.838427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:00.838955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:00.840653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:00.859184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 415/840] 10.222.0.25 () {56 vars in 1135 bytes} [Fri May 13 13:05:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:00.865123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:00.971671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22c5c2-8029-449d-b384-4bc60d321ee4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:01.006273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684bd45-23c8-478d-a49b-15373adbbe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:01.007300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684bd45-23c8-478d-a49b-15373adbbe1f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.007813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684bd45-23c8-478d-a49b-15373adbbe1f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.008289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684bd45-23c8-478d-a49b-15373adbbe1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.076530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684bd45-23c8-478d-a49b-15373adbbe1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:01.078359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684bd45-23c8-478d-a49b-15373adbbe1f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:01.175831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 426/841] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:01.182642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:01.283803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ade79-2eb6-4563-9874-22e9e2f9717d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:01.320642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d225b69-fcb0-453f-b3d7-179954a3b1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:01.321909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d225b69-fcb0-453f-b3d7-179954a3b1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.322402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d225b69-fcb0-453f-b3d7-179954a3b1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.322895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d225b69-fcb0-453f-b3d7-179954a3b1e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.334406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d225b69-fcb0-453f-b3d7-179954a3b1e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:01.336100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d225b69-fcb0-453f-b3d7-179954a3b1e7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5d225b69-fcb0-453f-b3d7-179954a3b1e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:01.355891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:01.356836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:01.357727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:01.357727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:01.357727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:01.357727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:01.360029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 416/842] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:01.365686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684bd45-23c8-478d-a49b-15373adbbe1f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:01.460526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c8e74-ac51-41e7-ae9c-4d7b9f36468b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.461052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c8e74-ac51-41e7-ae9c-4d7b9f36468b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.461052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c8e74-ac51-41e7-ae9c-4d7b9f36468b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.462771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 417/843] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:01.465519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684bd45-23c8-478d-a49b-15373adbbe1f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:01.467684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88827f24-b934-4fc5-b7d2-a959bd75d73f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.467975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88827f24-b934-4fc5-b7d2-a959bd75d73f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.468257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88827f24-b934-4fc5-b7d2-a959bd75d73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.490242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-88827f24-b934-4fc5-b7d2-a959bd75d73f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:01.491650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-88827f24-b934-4fc5-b7d2-a959bd75d73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:01.504342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:01.505147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.505991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.506394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.507231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:01.507641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:01.510113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:01.525391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 427/844] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:01.530699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:01.581932 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 418/845] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:01.597361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88827f24-b934-4fc5-b7d2-a959bd75d73f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:01.650130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72dd3f8-d7d8-45dd-a3c2-9ffc32b40c3d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:01.684792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:01.685603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.685923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.686227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.693330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:01.695028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:01.710194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:01.711133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:01.712095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:01.712095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:01.712095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:01.712095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:01.712095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:01.712095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:01.712095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:01.712095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:01.716193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 428/846] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:01.717214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:01.730109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88827f24-b934-4fc5-b7d2-a959bd75d73f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:01.773356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:01.774782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.775071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.775355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.776016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:01.777401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:01.789668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:01.791208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:01.792083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:01.796514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 419/847] 10.222.0.25 () {56 vars in 1132 bytes} [Fri May 13 13:05:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:01.797393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:01.828100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da367738-7fbc-42ac-b8da-73c1e984cf62 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:01.863288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8021ea-0bb6-421c-b73f-54d0a52b51b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:01.863887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8021ea-0bb6-421c-b73f-54d0a52b51b9 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.864176 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8021ea-0bb6-421c-b73f-54d0a52b51b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.864465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8021ea-0bb6-421c-b73f-54d0a52b51b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.865282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8021ea-0bb6-421c-b73f-54d0a52b51b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:01.865597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8021ea-0bb6-421c-b73f-54d0a52b51b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:01.867133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8021ea-0bb6-421c-b73f-54d0a52b51b9 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:01.885158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 429/848] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:01.885158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e153c-547c-4cbf-af13-811ba0e33701 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.885158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e153c-547c-4cbf-af13-811ba0e33701 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.885158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e153c-547c-4cbf-af13-811ba0e33701 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.888242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 430/849] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:01.889213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e153c-547c-4cbf-af13-811ba0e33701 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:01.911195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec505b-0560-4ea4-8d68-1d63ecd69b97 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:01.945954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5e096-3850-473f-835f-7c59bf3bf47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:01.946652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e096-3850-473f-835f-7c59bf3bf47a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.946934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e096-3850-473f-835f-7c59bf3bf47a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.947209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5e096-3850-473f-835f-7c59bf3bf47a None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.947913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5e096-3850-473f-835f-7c59bf3bf47a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:01.948218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5e096-3850-473f-835f-7c59bf3bf47a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:01.949583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5e096-3850-473f-835f-7c59bf3bf47a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:01.954114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 420/850] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:01.959939 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786fe9-90ac-42c3-bd30-0afa051e4f8e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:01.960334 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786fe9-90ac-42c3-bd30-0afa051e4f8e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:01.960714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786fe9-90ac-42c3-bd30-0afa051e4f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:01.977282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-89786fe9-90ac-42c3-bd30-0afa051e4f8e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:01.979158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-89786fe9-90ac-42c3-bd30-0afa051e4f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:01.991944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e153c-547c-4cbf-af13-811ba0e33701 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.023744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.024401 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.025208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.025486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.026349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=96f6d41c529444fba00bb09ac4224eaa) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:02.026659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:02.028080 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:02.034206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 431/851] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:01 2022] GET /identity/v3/role_assignments?user.id=96f6d41c529444fba00bb09ac4224eaa&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=663341777b0b44128ed78fe8d40f6462 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:02.034866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:02.058849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 421/852] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:02.060172 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8a0b6-36d6-4c93-82e0-5264a99c10a2 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.060462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8a0b6-36d6-4c93-82e0-5264a99c10a2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.060733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8a0b6-36d6-4c93-82e0-5264a99c10a2 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.062285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 422/853] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:02.067982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698d12b-667d-4540-8828-9e73e5d5a237 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.068310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698d12b-667d-4540-8828-9e73e5d5a237 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.068535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698d12b-667d-4540-8828-9e73e5d5a237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.085908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9698d12b-667d-4540-8828-9e73e5d5a237 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:02.087639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9698d12b-667d-4540-8828-9e73e5d5a237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:02.158624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 423/854] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:02.162608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f45afc-d56f-41d9-bf84-cfd3f0e933a2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.166805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698d12b-667d-4540-8828-9e73e5d5a237 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:02.201385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.202258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.202869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.203402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.204148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:02.204458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:02.206895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:02.221994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 432/855] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:02.221994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:02.290721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698d12b-667d-4540-8828-9e73e5d5a237 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.336863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.338341 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.338628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.338907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.339577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:02.340989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:02.353599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:02.354707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:02.355574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 424/856] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:02.361527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:02.371104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c925c-ccba-4d11-9463-510f86c1f4a7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.417412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feac04-a350-4949-817f-06a81e8491e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.418588 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feac04-a350-4949-817f-06a81e8491e9 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.419112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feac04-a350-4949-817f-06a81e8491e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.419590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feac04-a350-4949-817f-06a81e8491e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.436465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feac04-a350-4949-817f-06a81e8491e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-54feac04-a350-4949-817f-06a81e8491e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:02.444147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:02.445074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:05:02.445936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:02.449893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 433/857] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:02.452292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feac04-a350-4949-817f-06a81e8491e9 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:02.490801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c9a1e-1155-477d-bb50-7e7d6955adcf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.525286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.526144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.526517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.526919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.527646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:02.528651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:02.529446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:02.534628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 425/858] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:02.543850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:02.564036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feac04-a350-4949-817f-06a81e8491e9 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.606136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.607242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.607593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.608133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.608886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:02.609192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:02.610745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:02.624447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 434/859] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:02.628494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:02.651517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc50cb-0ec2-460f-a020-93c000777153 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.686433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.687323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.687323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.687582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.688260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:02.688566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:02.690202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:02.696532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 426/860] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:02.747827 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98493a5-3979-4da0-866a-43e1f061c6da None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.783617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.784433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.784712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.784984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.796958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:02.798622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:02.818085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:02.819043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:02.820025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:02.820025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:02.820025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:02.820025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:02.822483 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 435/861] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:02.828164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:02.937188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e814d4-6203-4a29-b5ca-29ae197152a3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:02.970732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:02.971459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:02.971745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:02.972025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:02.972749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:02.973113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:02.974521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:02.987067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 427/862] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:02.992198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:03.096884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa0aa-1db3-4977-a271-d5bc93532102 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:03.131157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:03.131977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:03.132258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:03.132530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:03.140183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:03.141633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:03.155946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:03.156867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:03.157847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 436/863] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:05:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:03.162820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:03.263414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9575164-c767-46b3-bf66-8b1f283e72d8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:03.296802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:03.297470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:03.297748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:03.298047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:03.298883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:03.299189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:03.300606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:03.322865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 428/864] 10.222.0.25 () {56 vars in 1149 bytes} [Fri May 13 13:05:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:03.326856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:03.429354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b315f-4c5b-46b7-bec3-d726a95821a0 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:03.464791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa1fec-4dd4-442b-b5fd-965abc5445c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:03.465880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1fec-4dd4-442b-b5fd-965abc5445c7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:03.466368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1fec-4dd4-442b-b5fd-965abc5445c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:03.466844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1fec-4dd4-442b-b5fd-965abc5445c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:03.540748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa1fec-4dd4-442b-b5fd-965abc5445c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:03.540748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa1fec-4dd4-442b-b5fd-965abc5445c7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:03.646764 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 437/865] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:03.648164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:03.744478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0d7b6-0fa3-493b-8f8d-0f1afaccb602 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:03.779104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:03.779889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:03.780169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:03.780442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:03.790877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:03.792248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:03.809466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:03.813814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:03.814733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:03.814733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:03.814733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:03.814733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:03.814733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 429/866] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:03.816614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa1fec-4dd4-442b-b5fd-965abc5445c7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:03.917619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa1fec-4dd4-442b-b5fd-965abc5445c7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:03.952534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:03.953201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:03.953482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:03.953758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:03.954519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:03.954822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:03.956195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:03.972080 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 438/867] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:03.973452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:04.077131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7ce12-258a-4446-bd39-53f445d735b9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:04.111741 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:04.112839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.113300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.113753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.120830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:04.122498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:04.136689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:04.137571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:04.138494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:04.138494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:04.138494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:04.138494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:04.138494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:04.138494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:04.138494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:04.138494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:04.140609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 430/868] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:04.146301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:04.253108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587c02a-7acc-415a-b260-ef13c12fbf89 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:04.287506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460246d-1da3-42d0-b111-7f4650446f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:04.288180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460246d-1da3-42d0-b111-7f4650446f5f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.288467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460246d-1da3-42d0-b111-7f4650446f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.288745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460246d-1da3-42d0-b111-7f4650446f5f None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.289533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460246d-1da3-42d0-b111-7f4650446f5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:04.289874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460246d-1da3-42d0-b111-7f4650446f5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:04.291367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460246d-1da3-42d0-b111-7f4650446f5f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:04.303989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 439/869] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:05:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:04.310510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:04.410012 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13581-4225-4ec5-a66d-7fe95ddd71ed None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.410533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13581-4225-4ec5-a66d-7fe95ddd71ed None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.410533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e13581-4225-4ec5-a66d-7fe95ddd71ed None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.412112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 440/870] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:04.417905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621800-74ad-4799-b163-967dfee44ba4 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.418197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621800-74ad-4799-b163-967dfee44ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.418482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621800-74ad-4799-b163-967dfee44ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.423811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a300-dbb3-4cdf-8618-7c7d618586ff None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:04.435387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-87621800-74ad-4799-b163-967dfee44ba4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:04.436561 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-87621800-74ad-4799-b163-967dfee44ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:04.458280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:04.459316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.459801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.460276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.461318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=63d11bc6b12949f28957146b7d66a71d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:04.461864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:04.464047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:04.470494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 431/871] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:04 2022] GET /identity/v3/role_assignments?user.id=63d11bc6b12949f28957146b7d66a71d&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:04.508668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 441/872] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:04.531661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:04.559817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8076ff-6c5e-428d-87d4-8ab14deb678c None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.560138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8076ff-6c5e-428d-87d4-8ab14deb678c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.560386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8076ff-6c5e-428d-87d4-8ab14deb678c None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.563246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 442/873] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:04.567721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d5d0-f0f8-4af1-8b7a-0f56f7d7a950 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.568019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d5d0-f0f8-4af1-8b7a-0f56f7d7a950 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.568280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d5d0-f0f8-4af1-8b7a-0f56f7d7a950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.592841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-3987d5d0-f0f8-4af1-8b7a-0f56f7d7a950 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:04.594091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3987d5d0-f0f8-4af1-8b7a-0f56f7d7a950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:04.659508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb13ce-7470-4df2-a499-d51dbf61cfd1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:04.687766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 443/874] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:04.703868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:04.704685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.704969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.705247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.707752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:04.707752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:04.709836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987d5d0-f0f8-4af1-8b7a-0f56f7d7a950 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:04.720937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:04.721976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:04.722966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 432/875] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:04.728542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:04.825683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987d5d0-f0f8-4af1-8b7a-0f56f7d7a950 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:04.854864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe38fd2-3cb6-4d11-ac99-41141e08f1d3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:04.874751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593db52e-bf67-43f1-a493-a120c899edeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:04.875973 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db52e-bf67-43f1-a493-a120c899edeb None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.876264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db52e-bf67-43f1-a493-a120c899edeb None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.876606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593db52e-bf67-43f1-a493-a120c899edeb None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.877519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593db52e-bf67-43f1-a493-a120c899edeb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:04.879624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593db52e-bf67-43f1-a493-a120c899edeb None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:04.894445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:04.895166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:04.895453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:04.895727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:04.896430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:04.896732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:04.898122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:04.902909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 433/876] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:04.908441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:05.016773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608b84-ae83-48b0-a0ef-4c45d1c87e19 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:05.017178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 444/877] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:05:04 2022] POST /identity/v3/services => generated 269 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:05.064208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:05.064466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f1045-aa12-4858-ae3f-6956564bd78b None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:05.064672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f1045-aa12-4858-ae3f-6956564bd78b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:05.064957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f1045-aa12-4858-ae3f-6956564bd78b None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:05.065617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:05.066072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:05.066379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 445/878] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:05.066708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:05.067649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:05.068113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:05.069755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:05.072540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93017e6c-b9e8-4c9a-955a-cdfa111ba5eb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:05.072831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93017e6c-b9e8-4c9a-955a-cdfa111ba5eb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:05.073112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93017e6c-b9e8-4c9a-955a-cdfa111ba5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:05.082089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 434/879] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:05.095630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-93017e6c-b9e8-4c9a-955a-cdfa111ba5eb None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:05.097019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-93017e6c-b9e8-4c9a-955a-cdfa111ba5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:05.173875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 446/880] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:05 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:05.198187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:05.319394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2375f-7a9f-4070-b189-7d876f391c4b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:05.354035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ecf11-2d45-4e93-9638-9600f15232d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:05.354766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ecf11-2d45-4e93-9638-9600f15232d3 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:05.355050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ecf11-2d45-4e93-9638-9600f15232d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:05.355370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ecf11-2d45-4e93-9638-9600f15232d3 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:05.356105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ecf11-2d45-4e93-9638-9600f15232d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:05.357481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ecf11-2d45-4e93-9638-9600f15232d3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:05.457741 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 435/881] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:05:05 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:05:06.884968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a834b5-43a1-4235-8b45-50ff56a0aabc None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:06.885471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a834b5-43a1-4235-8b45-50ff56a0aabc None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:06.885471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a834b5-43a1-4235-8b45-50ff56a0aabc None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:06.887067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 447/882] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:06.892515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28766de8-4f8a-473b-8f0a-d71b03ad6bb9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:06.893029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28766de8-4f8a-473b-8f0a-d71b03ad6bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:06.893192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28766de8-4f8a-473b-8f0a-d71b03ad6bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:06.909957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-28766de8-4f8a-473b-8f0a-d71b03ad6bb9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:06.911154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-28766de8-4f8a-473b-8f0a-d71b03ad6bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:06.993961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 436/883] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:07.016717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a834b5-43a1-4235-8b45-50ff56a0aabc None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.121898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a834b5-43a1-4235-8b45-50ff56a0aabc None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:07.159238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e145252-a185-4dde-bc0c-a776f46282fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:07.160446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e145252-a185-4dde-bc0c-a776f46282fe None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.160979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e145252-a185-4dde-bc0c-a776f46282fe None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.161460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e145252-a185-4dde-bc0c-a776f46282fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.174216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e145252-a185-4dde-bc0c-a776f46282fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-1e145252-a185-4dde-bc0c-a776f46282fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:07.182726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:07.183988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:07.184870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:07.187784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 448/884] 10.222.0.25 () {56 vars in 1120 bytes} [Fri May 13 13:05:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:07.193366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28766de8-4f8a-473b-8f0a-d71b03ad6bb9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.302772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28766de8-4f8a-473b-8f0a-d71b03ad6bb9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:07.336815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:07.338466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.338748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.339022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.339780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:07.340089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:07.340802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e7b30-feb2-4f1b-b9b5-7ef47f37c52e None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.341306 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e7b30-feb2-4f1b-b9b5-7ef47f37c52e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.341488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:07.341979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e7b30-feb2-4f1b-b9b5-7ef47f37c52e None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.343824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 449/885] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:07.349239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d83f34-db67-4846-85ce-27d527acd8f5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.349822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d83f34-db67-4846-85ce-27d527acd8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.350307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d83f34-db67-4846-85ce-27d527acd8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.350645 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 437/886] 10.222.0.25 () {56 vars in 1115 bytes} [Fri May 13 13:05:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:07.356524 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.369959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d8d83f34-db67-4846-85ce-27d527acd8f5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:07.371475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d8d83f34-db67-4846-85ce-27d527acd8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:07.445972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 450/887] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:07.448952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3ff9-b86f-4e72-a6bd-1a650c77c8a1 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.449490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3ff9-b86f-4e72-a6bd-1a650c77c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.449900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3ff9-b86f-4e72-a6bd-1a650c77c8a1 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.451606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 451/888] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:07.456635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3965e-cd17-4c9e-8186-994b2c633778 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:07.457349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb4937-66ff-4859-a969-3ab06d16664e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.457637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb4937-66ff-4859-a969-3ab06d16664e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.457985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb4937-66ff-4859-a969-3ab06d16664e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.478659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f6bb4937-66ff-4859-a969-3ab06d16664e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:07.479840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f6bb4937-66ff-4859-a969-3ab06d16664e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:07.501144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:07.501978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.502261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.502540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.513078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:07.514549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:07.531837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:07.532834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:07.533751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:07.533751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:07.533751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:07.533751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:07.533751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 438/889] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:07.534591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.551423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 452/890] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:07.552089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb4937-66ff-4859-a969-3ab06d16664e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.652069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681d15b-9cbf-43dc-8848-b66b9597b4cd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:07.655594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb4937-66ff-4859-a969-3ab06d16664e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:07.685592 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:07.686432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.686714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.686998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.687647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:07.689013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:07.693595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:07.694523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.694982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.695414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.696381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:07.696854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:07.698392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:07.701146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:07.702255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:07.703121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:07.707247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 439/891] 10.222.0.25 () {56 vars in 1132 bytes} [Fri May 13 13:05:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:07.708156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.710986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 453/892] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:07.716000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.804567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2acd6-efa3-4a16-8dc7-488de6233b4d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:07.812432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13984d-d655-47c5-a341-2f142ad1fb61 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:07.836378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82c39a-4be6-476d-9286-e72beb871669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:07.837063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82c39a-4be6-476d-9286-e72beb871669 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.837345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82c39a-4be6-476d-9286-e72beb871669 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.837619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82c39a-4be6-476d-9286-e72beb871669 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.838361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82c39a-4be6-476d-9286-e72beb871669 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:07.839723 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82c39a-4be6-476d-9286-e72beb871669 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:07.843631 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:07.851414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.851414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.851414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.851414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:07.851414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:07.851414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:07.851892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 454/893] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:07.852408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.931055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 440/894] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:05:07 2022] POST /identity/v3/services => generated 241 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:07.932550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27143d0b-2ce7-44aa-826e-2e23ce9552b9 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:07.932840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27143d0b-2ce7-44aa-826e-2e23ce9552b9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:07.933388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27143d0b-2ce7-44aa-826e-2e23ce9552b9 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:07.934720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 441/895] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:07.938971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27143d0b-2ce7-44aa-826e-2e23ce9552b9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:07.957687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc961dfc-1f2f-459c-8d35-477817612d4b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.001188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bbda5-39d2-4de2-8149-cffeddf146ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.003601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bbda5-39d2-4de2-8149-cffeddf146ff None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.003601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bbda5-39d2-4de2-8149-cffeddf146ff None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.003601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bbda5-39d2-4de2-8149-cffeddf146ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.011694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bbda5-39d2-4de2-8149-cffeddf146ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.013337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bbda5-39d2-4de2-8149-cffeddf146ff None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-e80bbda5-39d2-4de2-8149-cffeddf146ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:08.030077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:08.031344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:05:08.032594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:08.034551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 455/896] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:05:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:08.041055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213ebdd-f8ee-4501-a90b-af88bc3bd887 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.041055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213ebdd-f8ee-4501-a90b-af88bc3bd887 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.041055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213ebdd-f8ee-4501-a90b-af88bc3bd887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.047502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27143d0b-2ce7-44aa-826e-2e23ce9552b9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.054784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0213ebdd-f8ee-4501-a90b-af88bc3bd887 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:08.056146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0213ebdd-f8ee-4501-a90b-af88bc3bd887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:08.084280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.085062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.085346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.085625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.086378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:08.086685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.088631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.093976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 442/897] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:08.095039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.135522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 456/898] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:08.135522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213ebdd-f8ee-4501-a90b-af88bc3bd887 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.198665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d626921-56b4-440a-8e74-15e89a60f822 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.236356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213ebdd-f8ee-4501-a90b-af88bc3bd887 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.240742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.241519 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.241939 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.242318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.243289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:08.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.245240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.262574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 443/899] 10.222.0.25 () {56 vars in 1149 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:08.267719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.270133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.270871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.271225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.271565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.272438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] RBAC: Adding query filter params (interface=public, service_id=2af6e068e7014790aca2828d401b2480, region_id=RegionOne) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:08.272825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.274891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.285898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 457/900] 10.222.0.25 () {56 vars in 1251 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/endpoints?service_id=2af6e068e7014790aca2828d401b2480&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:08.285898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.378301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c042028-5ea3-4b4f-81da-077dda6f41b9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.403486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d71a05-5053-4ede-8efc-687832888b63 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.414268 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721674-9010-4090-a542-add624869736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.415687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721674-9010-4090-a542-add624869736 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.416096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721674-9010-4090-a542-add624869736 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.416470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721674-9010-4090-a542-add624869736 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.417221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29721674-9010-4090-a542-add624869736 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.418719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29721674-9010-4090-a542-add624869736 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-29721674-9010-4090-a542-add624869736 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:08.430423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:08.432198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:08.433471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:08.435244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 444/901] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:08.436099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721674-9010-4090-a542-add624869736 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.444425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d877fcc-7901-4ee0-bf22-548ac669fe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.445116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d877fcc-7901-4ee0-bf22-548ac669fe56 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.445400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d877fcc-7901-4ee0-bf22-548ac669fe56 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.445675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d877fcc-7901-4ee0-bf22-548ac669fe56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.519259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d877fcc-7901-4ee0-bf22-548ac669fe56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.520630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d877fcc-7901-4ee0-bf22-548ac669fe56 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.539235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721674-9010-4090-a542-add624869736 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.572153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c87b5-a39d-4144-a401-1e48bfa59934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.572824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c87b5-a39d-4144-a401-1e48bfa59934 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.573112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c87b5-a39d-4144-a401-1e48bfa59934 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.573393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c87b5-a39d-4144-a401-1e48bfa59934 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.574108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c87b5-a39d-4144-a401-1e48bfa59934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.575481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c87b5-a39d-4144-a401-1e48bfa59934 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.585985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 445/902] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:05:08.587318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c87b5-a39d-4144-a401-1e48bfa59934 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.625842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 458/903] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:08.626912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d877fcc-7901-4ee0-bf22-548ac669fe56 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.727895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c87b5-a39d-4144-a401-1e48bfa59934 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.753771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d877fcc-7901-4ee0-bf22-548ac669fe56 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.764885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.765623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.765864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.766150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.766868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:08.767177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.768574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.774170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 446/904] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:08.783084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.804926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.806694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.806977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.813517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.823416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.825549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:08.846739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:08.847671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:08.848835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:08.848835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:08.848835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:08.848835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:08.848835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 459/905] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:08.854382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.906587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236976a-49bd-4092-8469-e0962c5f3e64 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:08.940796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:08.941561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:08.941894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:08.942161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:08.942870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:08.943177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:08.944613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:08.949882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 447/906] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:08.956546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:08.972070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ab07a-660f-44ca-a69a-034d2d7c64ec None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:09.007117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:09.007804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:09.008090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:09.008417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:09.009186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:09.009501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:09.010930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:09.023857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 460/907] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:09.032815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:09.079140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5e338-0ecc-4766-96b5-3b7c04659a79 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:09.126876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:09.127638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:09.127989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:09.128332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:09.129201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] RBAC: Adding query filter params (interface=public, service_id=3f5a25dfdaa648e0be68ee78d0f572a7, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:09.129581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:09.131055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:09.136686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 448/908] 10.222.0.25 () {56 vars in 1251 bytes} [Fri May 13 13:05:08 2022] GET /identity/v3/endpoints?service_id=3f5a25dfdaa648e0be68ee78d0f572a7&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:09.141945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:09.146157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83858065-1d50-46f4-b0e3-c7151ce176ab None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:09.181565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:09.182465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:09.182809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:09.183372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:09.190645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:09.192219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:09.206627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:09.208675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:09.210853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:09.210853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:09.210853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:09.210853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:09.210853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:09.210853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:09.210853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:09.210853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:09.212851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 461/909] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:09.216261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:09.242079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f39ec5-f5bc-4f69-85e4-63cc62b506b5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:09.276882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b829f4-c117-4c83-a3d9-e30b28b7d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:09.277874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b829f4-c117-4c83-a3d9-e30b28b7d8bd None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:09.278169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b829f4-c117-4c83-a3d9-e30b28b7d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:09.278449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b829f4-c117-4c83-a3d9-e30b28b7d8bd None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:09.279396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b829f4-c117-4c83-a3d9-e30b28b7d8bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:09.280836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b829f4-c117-4c83-a3d9-e30b28b7d8bd None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:09.294069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 449/910] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:05:09 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:05:09.320913 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b62cfe-e3aa-4ad4-a3e8-19280e60b35b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:09.356943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b2248-26b7-4ca8-b2a2-e40e7eff5cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:09.357640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b2248-26b7-4ca8-b2a2-e40e7eff5cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:09.357959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b2248-26b7-4ca8-b2a2-e40e7eff5cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:09.358239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b2248-26b7-4ca8-b2a2-e40e7eff5cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:09.359060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b2248-26b7-4ca8-b2a2-e40e7eff5cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:09.359381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b2248-26b7-4ca8-b2a2-e40e7eff5cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:09.360879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b2248-26b7-4ca8-b2a2-e40e7eff5cd4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:09.375457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 462/911] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:09.381462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b829f4-c117-4c83-a3d9-e30b28b7d8bd None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:09.501876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b829f4-c117-4c83-a3d9-e30b28b7d8bd None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:09.535132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7212aa-d96d-46b3-a6a9-53d291f742d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:09.535792 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7212aa-d96d-46b3-a6a9-53d291f742d0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:09.536076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7212aa-d96d-46b3-a6a9-53d291f742d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:09.536357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7212aa-d96d-46b3-a6a9-53d291f742d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:09.537196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7212aa-d96d-46b3-a6a9-53d291f742d0 None admin] RBAC: Adding query filter params (role.id=4db664dca3c640eea82b75a3e905becd, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=63d11bc6b12949f28957146b7d66a71d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:09.537508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7212aa-d96d-46b3-a6a9-53d291f742d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:09.538990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7212aa-d96d-46b3-a6a9-53d291f742d0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:09.545566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 450/912] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:09 2022] GET /identity/v3/role_assignments?user.id=63d11bc6b12949f28957146b7d66a71d&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=4db664dca3c640eea82b75a3e905becd => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:10.242393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f8afb-b5bf-4504-8272-10e6e344886d None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:10.243183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f8afb-b5bf-4504-8272-10e6e344886d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:10.243183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f8afb-b5bf-4504-8272-10e6e344886d None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:10.244589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 463/913] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:10.249933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593824b2-d876-4c3a-b131-0a5a774ebc18 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:10.250394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593824b2-d876-4c3a-b131-0a5a774ebc18 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:10.250574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593824b2-d876-4c3a-b131-0a5a774ebc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:10.267084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-593824b2-d876-4c3a-b131-0a5a774ebc18 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:10.268274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-593824b2-d876-4c3a-b131-0a5a774ebc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:10.335137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 451/914] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:10.358552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f8afb-b5bf-4504-8272-10e6e344886d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:10.462857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f8afb-b5bf-4504-8272-10e6e344886d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:10.508157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:10.508933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:10.509209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:10.509476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:10.510182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:10.511574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:10.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:10.526107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:10.526931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 464/915] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:10.534229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593824b2-d876-4c3a-b131-0a5a774ebc18 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:10.637192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593824b2-d876-4c3a-b131-0a5a774ebc18 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:10.673859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:10.674924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:10.675424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:10.675919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:10.676838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:10.677346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:10.678955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:10.684961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 452/916] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:10.690740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:10.800844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33a8d8-20dc-4ffb-89a5-37b6af8ed0cd None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:10.834314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637594f-97c2-4dc9-9ca0-1d81058f58fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:10.834979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637594f-97c2-4dc9-9ca0-1d81058f58fd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:10.835257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637594f-97c2-4dc9-9ca0-1d81058f58fd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:10.835538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637594f-97c2-4dc9-9ca0-1d81058f58fd None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:10.836238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637594f-97c2-4dc9-9ca0-1d81058f58fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:10.836592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637594f-97c2-4dc9-9ca0-1d81058f58fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:10.838123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637594f-97c2-4dc9-9ca0-1d81058f58fd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:10.843341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 465/917] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:10.849728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:10.952203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecc9a1-ef9a-442f-9d8b-e97cee55c13b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:10.952671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c57d-29a9-4c86-b380-31eb21d0adcd None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:10.952965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c57d-29a9-4c86-b380-31eb21d0adcd None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:10.953251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c57d-29a9-4c86-b380-31eb21d0adcd None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:10.954928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 466/918] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:10.959973 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6a9f-b89b-4c58-9347-0431c0362357 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:10.960256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6a9f-b89b-4c58-9347-0431c0362357 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:10.960533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6a9f-b89b-4c58-9347-0431c0362357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:10.976989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-a3ad6a9f-b89b-4c58-9347-0431c0362357 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:10.978238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a3ad6a9f-b89b-4c58-9347-0431c0362357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:10.988903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:10.989883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:10.990412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:10.990889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:10.991869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] RBAC: Adding query filter params (interface=public, service_id=1b5d99bc3e754126aa215f265f68247e, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:10.992379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:10.994506 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:11.000088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 453/919] 10.222.0.25 () {56 vars in 1251 bytes} [Fri May 13 13:05:10 2022] GET /identity/v3/endpoints?service_id=1b5d99bc3e754126aa215f265f68247e&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:11.005650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:11.051323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 467/920] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:11.069118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad6a9f-b89b-4c58-9347-0431c0362357 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:11.106828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570fcf1-448a-4838-9bd9-5b392f192cef None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:11.140560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e848-a5d0-471e-b7cd-fda5567c34b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:11.141602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e848-a5d0-471e-b7cd-fda5567c34b5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.142109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e848-a5d0-471e-b7cd-fda5567c34b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.142606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0e848-a5d0-471e-b7cd-fda5567c34b5 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.143509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0e848-a5d0-471e-b7cd-fda5567c34b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:11.145201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0e848-a5d0-471e-b7cd-fda5567c34b5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:11.152354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 454/921] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:05:11 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:05:11.171725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad6a9f-b89b-4c58-9347-0431c0362357 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:11.209903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:11.210742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.211080 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.211357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.212607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:11.214048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:11.227214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:11.228773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 13 13:05:11.229608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:11.233907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 468/922] 10.222.0.25 () {56 vars in 1132 bytes} [Fri May 13 13:05:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:11.235658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e848-a5d0-471e-b7cd-fda5567c34b5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:11.345061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0e848-a5d0-471e-b7cd-fda5567c34b5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:11.380840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:11.381895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.382386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.382892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.383830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:11.384349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:11.385952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:11.391449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 455/923] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:11.397019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:11.531506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78f19-39a9-4672-84db-778c4cb77926 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:11.566151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd06d5-f588-4961-89eb-349e178fa99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:11.566819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd06d5-f588-4961-89eb-349e178fa99f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.567148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd06d5-f588-4961-89eb-349e178fa99f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.567428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd06d5-f588-4961-89eb-349e178fa99f None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.568134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd06d5-f588-4961-89eb-349e178fa99f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:11.568443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd06d5-f588-4961-89eb-349e178fa99f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:11.569824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd06d5-f588-4961-89eb-349e178fa99f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:11.579490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 469/924] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:11.581076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:11.660192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11736f-216e-453b-9c55-5ba0a91500d8 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.660398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11736f-216e-453b-9c55-5ba0a91500d8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.660937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11736f-216e-453b-9c55-5ba0a91500d8 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.662789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 470/925] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:11.668150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c1bf-a116-4874-b540-0bfd7ce65440 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.668674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c1bf-a116-4874-b540-0bfd7ce65440 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.669165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c1bf-a116-4874-b540-0bfd7ce65440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.687577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6f54c1bf-a116-4874-b540-0bfd7ce65440 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:11.689141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6f54c1bf-a116-4874-b540-0bfd7ce65440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:11.690185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cc917-a7ba-489c-a089-9b92731672cc None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:11.726395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036ccbf-1562-4889-a6a1-f17227eb22f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:11.727066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ccbf-1562-4889-a6a1-f17227eb22f1 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.727383 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ccbf-1562-4889-a6a1-f17227eb22f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.727662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ccbf-1562-4889-a6a1-f17227eb22f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.728410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036ccbf-1562-4889-a6a1-f17227eb22f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:11.729780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036ccbf-1562-4889-a6a1-f17227eb22f1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:11.757849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 471/926] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:11.781044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c1bf-a116-4874-b540-0bfd7ce65440 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:11.824505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 456/927] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:05:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:11.826556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64aff8-7d1b-4e7c-a9b6-2f5e2be0e202 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.827128 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64aff8-7d1b-4e7c-a9b6-2f5e2be0e202 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.827613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64aff8-7d1b-4e7c-a9b6-2f5e2be0e202 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.833715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 457/928] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:11.847613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fada48-95b0-4703-aab1-2a10b394ca5f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.847912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fada48-95b0-4703-aab1-2a10b394ca5f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.848187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fada48-95b0-4703-aab1-2a10b394ca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.880775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-75fada48-95b0-4703-aab1-2a10b394ca5f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:11.883023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-75fada48-95b0-4703-aab1-2a10b394ca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:11.913141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c1bf-a116-4874-b540-0bfd7ce65440 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:11.968462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:11.969543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:11.970002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:11.970428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:11.971311 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:11.972925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:11.981343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 458/929] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:11.987666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:11.988858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 13 13:05:11.989718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:11.992341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 472/930] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:11.998060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fada48-95b0-4703-aab1-2a10b394ca5f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:12.006546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:12.101223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fada48-95b0-4703-aab1-2a10b394ca5f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:12.120402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8a3da-0f0e-4998-a04f-43cb1bebf235 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:12.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:12.136942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:12.137389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:12.137823 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:12.138662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:12.139108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:12.140610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:12.145755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 459/931] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:12.151117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:12.163313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a8168-5acf-407b-b29e-2f495ae10743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:12.165508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a8168-5acf-407b-b29e-2f495ae10743 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:12.165976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a8168-5acf-407b-b29e-2f495ae10743 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:12.166418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a8168-5acf-407b-b29e-2f495ae10743 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:12.179872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a8168-5acf-407b-b29e-2f495ae10743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-7d1a8168-5acf-407b-b29e-2f495ae10743 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:12.191478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:12.192620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:12.193633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:12.196394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 473/932] 10.222.0.25 () {56 vars in 1120 bytes} [Fri May 13 13:05:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:12.204460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a8168-5acf-407b-b29e-2f495ae10743 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:12.259579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d87efa-a3b0-4344-b822-bc1e8dbf242a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:12.298037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:12.299014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:12.299503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:12.299955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:12.301357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:12.301880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:12.303961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:12.312166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 460/933] 10.222.0.25 () {56 vars in 1114 bytes} [Fri May 13 13:05:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:12.317959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:12.328571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a8168-5acf-407b-b29e-2f495ae10743 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:12.364340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:12.365013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:12.365303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:12.365582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:12.366414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:12.366723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:12.368661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:12.381532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 474/934] 10.222.0.25 () {56 vars in 1115 bytes} [Fri May 13 13:05:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:12.383254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:12.446538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883076a-820c-45b1-b1ab-1f8bb3fd068a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:12.490390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34ca4-a41f-4ca1-9716-3465d84906e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:12.491687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34ca4-a41f-4ca1-9716-3465d84906e9 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:12.492411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34ca4-a41f-4ca1-9716-3465d84906e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:12.492841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34ca4-a41f-4ca1-9716-3465d84906e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:12.493972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34ca4-a41f-4ca1-9716-3465d84906e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:12.495502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34ca4-a41f-4ca1-9716-3465d84906e9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:12.508920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3619a0-c161-457d-8084-baaafe7a514d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:12.542753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:12.543549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:12.543832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:12.544107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:12.554426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:12.556429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:12.574060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:12.574953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:12.575874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:12.575874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:12.575874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:12.575874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:12.575874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 475/935] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:12.581816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:12.600749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 461/936] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:05:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:12.699183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a280-931b-4b95-b2f1-215fcc3812d7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:12.741498 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:12.742703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:12.743724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:12.744208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:12.745178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:12.745697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:12.747392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:12.765866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 476/937] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:12.767867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34ca4-a41f-4ca1-9716-3465d84906e9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:12.870722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34ca4-a41f-4ca1-9716-3465d84906e9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:12.903912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:12.905020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:12.905306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:12.905585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:12.913205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:12.914685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:12.929836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:12.931355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:12.932313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 462/938] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:05:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:12.937739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:13.040280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a69c9-a3ff-49da-a5a6-e07f1b314f8b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:13.074982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:13.075666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.075949 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.076224 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.077062 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:13.077370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:13.078836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:13.097027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 477/939] 10.222.0.25 () {56 vars in 1149 bytes} [Fri May 13 13:05:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:13.103114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:13.206016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc8bf2-5b1e-49bd-8628-d6f1b55012d1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:13.238482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:13.239286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.239565 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.239836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.252823 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:13.254323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:13.274964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:13.276459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:13.277996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:13.277996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:13.277996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:13.277996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:13.277996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 463/940] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:13.287593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:13.406217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb1b7-6c29-4069-a20a-e7809df3dd30 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:13.440593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dba19-1fd1-48e6-a3c8-af94ab707748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:13.441653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dba19-1fd1-48e6-a3c8-af94ab707748 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.442172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dba19-1fd1-48e6-a3c8-af94ab707748 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.442668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dba19-1fd1-48e6-a3c8-af94ab707748 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.443584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dba19-1fd1-48e6-a3c8-af94ab707748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:13.444091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dba19-1fd1-48e6-a3c8-af94ab707748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:13.445676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dba19-1fd1-48e6-a3c8-af94ab707748 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:13.458232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 478/941] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:13.464639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:13.471942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572cfc-8536-491d-a335-f4e26da8a8ca None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.472236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572cfc-8536-491d-a335-f4e26da8a8ca None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.472516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572cfc-8536-491d-a335-f4e26da8a8ca None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.474136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 479/942] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:13.484905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd6cf3-ed15-4aa9-bf46-97b604491cbc None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.485398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd6cf3-ed15-4aa9-bf46-97b604491cbc None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.485893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd6cf3-ed15-4aa9-bf46-97b604491cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.502795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-19dd6cf3-ed15-4aa9-bf46-97b604491cbc None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:13.504305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-19dd6cf3-ed15-4aa9-bf46-97b604491cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:13.577809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f2a1f-057d-4390-99fb-73014006b3c4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:13.578391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 480/943] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:13.601480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd6cf3-ed15-4aa9-bf46-97b604491cbc None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:13.609945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:13.610754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.611036 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.611319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.619132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:13.620665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:13.637107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:13.638083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:13.638974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 464/944] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:13.644831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:13.740149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd6cf3-ed15-4aa9-bf46-97b604491cbc None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:13.768023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb776642-ad9b-42c9-8cf5-8f9b1048b5b4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:13.782947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:13.783990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.784348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.784703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.785599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:13.787743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:13.805100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:13.814529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:13.815548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 481/945] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:13.816495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:13.816495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.816495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.816495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.816882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:13.816971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:13.816971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:13.817217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:13.829448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 465/946] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:13.834913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:13.944564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b1b68-bd0f-4509-b7b4-27d3f3ad5ded None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:13.958396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30092b7f-ecef-474b-85ff-4d51fe9da687 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:13.993144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:13.994213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.994680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.995132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:13.996248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:13.996733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:13.997182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9dbd3-5c67-4477-a6ca-3685a5d96f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:13.998344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9dbd3-5c67-4477-a6ca-3685a5d96f3c None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:13.998506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:13.998890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9dbd3-5c67-4477-a6ca-3685a5d96f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:13.999349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9dbd3-5c67-4477-a6ca-3685a5d96f3c None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/63d11bc6b12949f28957146b7d66a71d/roles/4db664dca3c640eea82b75a3e905becd` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.001825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9dbd3-5c67-4477-a6ca-3685a5d96f3c None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=63d11bc6b12949f28957146b7d66a71d, role_id=4db664dca3c640eea82b75a3e905becd)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:14.003472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9dbd3-5c67-4477-a6ca-3685a5d96f3c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:14.005865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 482/947] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:14.010130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:14.094750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 466/948] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:05:13 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/63d11bc6b12949f28957146b7d66a71d/roles/4db664dca3c640eea82b75a3e905becd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:05:14.121145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee98d63-8da0-4f12-8e67-51d4b4c6f10c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:14.141520 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e110bc-e66c-4913-b38a-a8a8afbc64e3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.141852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e110bc-e66c-4913-b38a-a8a8afbc64e3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.142109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e110bc-e66c-4913-b38a-a8a8afbc64e3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.143464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 467/949] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:14.149029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffca019-2af6-49c2-ad52-aeea5131b08a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.149836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffca019-2af6-49c2-ad52-aeea5131b08a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.150118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffca019-2af6-49c2-ad52-aeea5131b08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.159813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:14.160553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.160886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.161213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.162584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:14.162897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:14.164305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:14.171431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 483/950] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:14.171829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dffca019-2af6-49c2-ad52-aeea5131b08a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:14.173178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dffca019-2af6-49c2-ad52-aeea5131b08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:14.179123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:14.271246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 468/951] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:14.293601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffca019-2af6-49c2-ad52-aeea5131b08a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:14.294630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32325c1-d996-447a-a972-f5cfc8c3159a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:14.341760 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15809f06-3563-4ced-aef5-a2880cbe3daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:14.342565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15809f06-3563-4ced-aef5-a2880cbe3daf None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.342766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15809f06-3563-4ced-aef5-a2880cbe3daf None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.343081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15809f06-3563-4ced-aef5-a2880cbe3daf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.343818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15809f06-3563-4ced-aef5-a2880cbe3daf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:14.345236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15809f06-3563-4ced-aef5-a2880cbe3daf None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:14.398832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffca019-2af6-49c2-ad52-aeea5131b08a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:14.432464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:14.433519 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.433867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.434366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.435060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:14.436532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:14.452960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:14.454046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:14.455462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 469/952] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:14.463263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:14.469927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 484/953] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:05:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:05:14.577267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d3917-d088-4eae-84ba-540777e9e711 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:14.619821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53f42e-2486-4dd6-af02-0ef16ca0ebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:14.620497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53f42e-2486-4dd6-af02-0ef16ca0ebc4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.620775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53f42e-2486-4dd6-af02-0ef16ca0ebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.621045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53f42e-2486-4dd6-af02-0ef16ca0ebc4 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.621756 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53f42e-2486-4dd6-af02-0ef16ca0ebc4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:14.622100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53f42e-2486-4dd6-af02-0ef16ca0ebc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:14.623500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53f42e-2486-4dd6-af02-0ef16ca0ebc4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:14.633745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 470/954] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:05:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:14.635235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15809f06-3563-4ced-aef5-a2880cbe3daf None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:14.741204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15809f06-3563-4ced-aef5-a2880cbe3daf None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:14.775564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2aac79-1835-4572-a31a-4b553789143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:14.776245 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aac79-1835-4572-a31a-4b553789143d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.776531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aac79-1835-4572-a31a-4b553789143d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.776809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aac79-1835-4572-a31a-4b553789143d None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.777523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2aac79-1835-4572-a31a-4b553789143d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:14.777857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2aac79-1835-4572-a31a-4b553789143d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:14.779240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2aac79-1835-4572-a31a-4b553789143d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:14.784319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 485/955] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:05:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:14.988579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c922fa-7cc4-4ec5-8435-838560abfb2a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.988973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c922fa-7cc4-4ec5-8435-838560abfb2a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.989167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c922fa-7cc4-4ec5-8435-838560abfb2a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:14.990788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 471/956] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:14.997045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3791984-38ab-4c6b-b9d3-3e987762f2fe None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:14.997348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3791984-38ab-4c6b-b9d3-3e987762f2fe None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:14.997634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3791984-38ab-4c6b-b9d3-3e987762f2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:15.014957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d3791984-38ab-4c6b-b9d3-3e987762f2fe None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:15.016156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d3791984-38ab-4c6b-b9d3-3e987762f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:15.082552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 486/957] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:15.105840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c922fa-7cc4-4ec5-8435-838560abfb2a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:15.209283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c922fa-7cc4-4ec5-8435-838560abfb2a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:15.243110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:15.243705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:15.243991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:15.244271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:15.244985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:15.246516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:15.247884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING keystone.server.flask.common [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 13 13:05:15.290800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.sql.core [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 13:05:15.290800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 13 13:05:15.290800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [parameters: {'id': 'dc3cd05d61634362912f0537aabf2c08', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 13 13:05:15.290800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80277) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc3cd05d61634362912f0537aabf2c08.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc3cd05d61634362912f0537aabf2c08. May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application conn.query(q) May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:05:15.309003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result.read() May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 13 13:05:15.309912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return project_ref.to_dict() May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application next(self.gen) May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application yield resource May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application next(self.gen) May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.session.rollback() May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.force_reraise() May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise self.value May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application session.commit() May 13 13:05:15.311428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._prepare_impl() May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.session.flush() May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._flush(objects) May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application compat.raise_( May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application flush_context.execute() May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 13:05:15.313607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rec.execute(self) May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application c = connection._execute_20( May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 13:05:15.317731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application conn.query(q) May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result.read() May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:05:15.319403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [parameters: {'id': 'dc3cd05d61634362912f0537aabf2c08', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:15.320901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:15.328013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.Conflict( May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc3cd05d61634362912f0537aabf2c08. May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:15.330174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 472/958] 10.222.0.25 () {60 vars in 1140 bytes} [Fri May 13 13:05:15 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:15.331408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3791984-38ab-4c6b-b9d3-3e987762f2fe None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:15.420480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3791984-38ab-4c6b-b9d3-3e987762f2fe None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:15.457690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f1461-afdc-4334-88f9-f5c03f27678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:15.458519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f1461-afdc-4334-88f9-f5c03f27678e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:15.458802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f1461-afdc-4334-88f9-f5c03f27678e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:15.459076 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f1461-afdc-4334-88f9-f5c03f27678e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:15.465733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f1461-afdc-4334-88f9-f5c03f27678e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:15.467205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f1461-afdc-4334-88f9-f5c03f27678e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-839f1461-afdc-4334-88f9-f5c03f27678e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:15.486163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:15.487064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:15.488150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 487/959] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:15.493998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:15.595137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a2557-5cd0-4689-9eff-77cb06571ad7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:15.629033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b829da-6e04-455c-afae-3acf75927fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:15.629702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b829da-6e04-455c-afae-3acf75927fdb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:15.630026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b829da-6e04-455c-afae-3acf75927fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:15.630307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b829da-6e04-455c-afae-3acf75927fdb None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:15.631070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b829da-6e04-455c-afae-3acf75927fdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:15.631376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b829da-6e04-455c-afae-3acf75927fdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:15.633015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b829da-6e04-455c-afae-3acf75927fdb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:15.657568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 473/960] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:05:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:16.427035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94248f-64e5-44f3-82df-3bd4ab879302 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:16.427674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94248f-64e5-44f3-82df-3bd4ab879302 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:16.427674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94248f-64e5-44f3-82df-3bd4ab879302 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:16.429229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 488/961] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:16.434751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114cdac-dabd-4876-a3bc-06f2edf1c002 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:16.435044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114cdac-dabd-4876-a3bc-06f2edf1c002 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:16.435324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114cdac-dabd-4876-a3bc-06f2edf1c002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:16.453006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6114cdac-dabd-4876-a3bc-06f2edf1c002 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:16.454530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6114cdac-dabd-4876-a3bc-06f2edf1c002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:16.547081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 474/962] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:16.570222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94248f-64e5-44f3-82df-3bd4ab879302 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:16.678432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94248f-64e5-44f3-82df-3bd4ab879302 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:16.712227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c176a-be42-4e2f-8498-438c82f480fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:16.713066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c176a-be42-4e2f-8498-438c82f480fa None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:16.713352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c176a-be42-4e2f-8498-438c82f480fa None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:16.713629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c176a-be42-4e2f-8498-438c82f480fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:16.724073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c176a-be42-4e2f-8498-438c82f480fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-604c176a-be42-4e2f-8498-438c82f480fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:16.734027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:16.736134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:16.738193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 489/963] 10.222.0.25 () {56 vars in 1120 bytes} [Fri May 13 13:05:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:16.743671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114cdac-dabd-4876-a3bc-06f2edf1c002 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:16.816731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15847b-6c34-4706-a1c4-acdcfb233424 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:16.817012 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15847b-6c34-4706-a1c4-acdcfb233424 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:16.817468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15847b-6c34-4706-a1c4-acdcfb233424 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:16.823886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 490/964] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:16.825557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15341af-4c43-4ba4-b082-5ea7ca2ceb64 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:16.826339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15341af-4c43-4ba4-b082-5ea7ca2ceb64 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:16.826693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15341af-4c43-4ba4-b082-5ea7ca2ceb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:16.845023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114cdac-dabd-4876-a3bc-06f2edf1c002 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:16.848262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b15341af-4c43-4ba4-b082-5ea7ca2ceb64 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:16.849522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b15341af-4c43-4ba4-b082-5ea7ca2ceb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:16.884445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:16.885145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:16.885430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:16.885706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:16.886526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:16.886835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:16.888258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:16.897742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 475/965] 10.222.0.25 () {56 vars in 1115 bytes} [Fri May 13 13:05:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:16.906879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:16.946241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 491/966] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:16.968290 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15341af-4c43-4ba4-b082-5ea7ca2ceb64 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.022205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ef238-8285-4be5-b08b-32c4b4a86b3b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.059558 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e3071-3b85-4c76-bd8c-5b7e35988ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.060707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3071-3b85-4c76-bd8c-5b7e35988ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.061186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3071-3b85-4c76-bd8c-5b7e35988ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.061681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3071-3b85-4c76-bd8c-5b7e35988ca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.073076 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15341af-4c43-4ba4-b082-5ea7ca2ceb64 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.073925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e3071-3b85-4c76-bd8c-5b7e35988ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.075625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e3071-3b85-4c76-bd8c-5b7e35988ca6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-379e3071-3b85-4c76-bd8c-5b7e35988ca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:17.092702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:17.093830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:17.094759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:17.094759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:17.094759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:17.094759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:17.094759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 476/967] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:17.095765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e600e-8dd6-49e4-9695-3ce4d6386df7 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.095765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e600e-8dd6-49e4-9695-3ce4d6386df7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.096027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e600e-8dd6-49e4-9695-3ce4d6386df7 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.097530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 477/968] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:17.099270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e600e-8dd6-49e4-9695-3ce4d6386df7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.112759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbb72d-5b63-4643-aefe-e0fd37fca218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.114142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbb72d-5b63-4643-aefe-e0fd37fca218 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.114865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbb72d-5b63-4643-aefe-e0fd37fca218 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.115295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbb72d-5b63-4643-aefe-e0fd37fca218 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.116254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbb72d-5b63-4643-aefe-e0fd37fca218 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.118519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbb72d-5b63-4643-aefe-e0fd37fca218 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-41cbb72d-5b63-4643-aefe-e0fd37fca218 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:17.132208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:17.133249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:17.134573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 492/969] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:17.135392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb79d-abba-4c09-a699-ce9cc18fef4f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.135392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb79d-abba-4c09-a699-ce9cc18fef4f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.135532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb79d-abba-4c09-a699-ce9cc18fef4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.153182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-524fb79d-abba-4c09-a699-ce9cc18fef4f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:17.154500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-524fb79d-abba-4c09-a699-ce9cc18fef4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:17.204917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e600e-8dd6-49e4-9695-3ce4d6386df7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.221763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 493/970] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:17.222805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fb79d-abba-4c09-a699-ce9cc18fef4f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.242488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.243199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.243475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.243754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.244471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:17.244781 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.246275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.258145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 478/971] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:17.259215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.335151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fb79d-abba-4c09-a699-ce9cc18fef4f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.355349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c11fb-3a3b-43b9-83bc-1ea02758c169 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.372961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.373634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.373957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.374235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.374943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:17.375251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.376624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.381449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 494/972] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:17.382254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.395699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420c7d-0124-400a-9e49-fd375e5794a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.396448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420c7d-0124-400a-9e49-fd375e5794a4 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.396793 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420c7d-0124-400a-9e49-fd375e5794a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.397135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420c7d-0124-400a-9e49-fd375e5794a4 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.397956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60420c7d-0124-400a-9e49-fd375e5794a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.400071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60420c7d-0124-400a-9e49-fd375e5794a4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.477533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0946048-9258-4e61-bb67-00e3cd635b1a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.510849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.511673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.511966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.512243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.514768 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 479/973] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:05:17 2022] POST /identity/v3/services => generated 242 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:17.519957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.534646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.534800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420c7d-0124-400a-9e49-fd375e5794a4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:17.544620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:17.545958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:17.547130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 495/974] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:05:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:17.553418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.659596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420c7d-0124-400a-9e49-fd375e5794a4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.663680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54d344-b62c-4a1b-bec6-c03117eb3cb0 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.701729 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.702588 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.702939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.703286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.704135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:17.704457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.705935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.707131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.709192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.709192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.709447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.710205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:17.710514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.712500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.725034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 480/975] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:17.726456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.729726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 496/976] 10.222.0.25 () {56 vars in 1149 bytes} [Fri May 13 13:05:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:17.735572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.840542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbd5ee-794f-4a3a-8956-18af42dcb5da None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.866327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19706b1-5897-488d-9690-d1c73f1b32fd None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:17.880789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.881455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.881743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.882052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.882904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] RBAC: Adding query filter params (interface=admin, service_id=1b5d99bc3e754126aa215f265f68247e, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:17.883168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.884579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:17.890521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 481/977] 10.222.0.25 () {56 vars in 1249 bytes} [Fri May 13 13:05:17 2022] GET /identity/v3/endpoints?service_id=1b5d99bc3e754126aa215f265f68247e&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:17.896001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:17.915538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779ef58-d024-4af1-8160-57786646a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:17.916676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779ef58-d024-4af1-8160-57786646a946 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:17.917128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779ef58-d024-4af1-8160-57786646a946 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:17.917906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779ef58-d024-4af1-8160-57786646a946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:17.992707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779ef58-d024-4af1-8160-57786646a946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:17.994154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779ef58-d024-4af1-8160-57786646a946 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.001820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ee545-3b5f-4641-9ebe-d05d4422bf9d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:18.034634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535f8c5-e1c2-4d30-a6a3-816e044c4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:18.035302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535f8c5-e1c2-4d30-a6a3-816e044c4ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.035576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535f8c5-e1c2-4d30-a6a3-816e044c4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.035858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535f8c5-e1c2-4d30-a6a3-816e044c4ea3 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.036518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535f8c5-e1c2-4d30-a6a3-816e044c4ea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:18.037904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535f8c5-e1c2-4d30-a6a3-816e044c4ea3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.046130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 482/978] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:05:17 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:05:18.047739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5815934-9523-4145-8312-412a1804c80e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.048447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5815934-9523-4145-8312-412a1804c80e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.048846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5815934-9523-4145-8312-412a1804c80e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.051370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 483/979] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:18.059862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2de1c-9c04-41e1-afec-cc5b4c393674 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.060394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2de1c-9c04-41e1-afec-cc5b4c393674 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.060832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2de1c-9c04-41e1-afec-cc5b4c393674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.079796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a1d2de1c-9c04-41e1-afec-cc5b4c393674 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:18.081313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a1d2de1c-9c04-41e1-afec-cc5b4c393674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:18.111664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 497/980] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:17 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:18.118642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779ef58-d024-4af1-8160-57786646a946 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:18.173014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 484/981] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:18.197915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2de1c-9c04-41e1-afec-cc5b4c393674 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:18.226863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779ef58-d024-4af1-8160-57786646a946 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:18.267908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:18.268704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.268986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.269408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.283748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:18.285233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:18.308828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:18.309817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:18.310739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:18.310739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:18.310739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:18.310739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:18.310739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 498/982] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:18.315504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:18.321439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2de1c-9c04-41e1-afec-cc5b4c393674 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:18.357181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:18.358260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.358756 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.359218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.360118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:18.361738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.413847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c7417-68b1-41e1-bf27-10e965027e4d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:18.427403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.sql.core [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 13:05:18.427403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 13 13:05:18.427403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [parameters: {'user_id': 'c75d89309ba341019a156cb57da2ba17', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 13 13:05:18.427403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80277) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application conn.query(q) May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result.read() May 13 13:05:18.435705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:05:18.443585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application next(self.gen) May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application yield resource May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application next(self.gen) May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.session.rollback() May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.force_reraise() May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise self.value May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application session.commit() May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 13 13:05:18.444580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._prepare_impl() May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.session.flush() May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._flush(objects) May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application compat.raise_( May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application flush_context.execute() May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rec.execute(self) May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 13 13:05:18.445533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application _emit_insert_statements( May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = connection._execute_20( May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ret = self._execute_context( May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 13 13:05:18.448356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self.dialect.do_execute( May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result = self._query(query) May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application conn.query(q) May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application result.read() May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 13 13:05:18.449229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application packet.raise_for_error() May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c75d89309ba341019a156cb57da2ba17', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:18.450122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 13 13:05:18.451762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 13 13:05:18.454516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 13 13:05:18.454516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 13 13:05:18.454516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 13 13:05:18.454516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 13 13:05:18.454516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 13 13:05:18.454516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:18.454516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 485/983] 10.222.0.25 () {60 vars in 1131 bytes} [Fri May 13 13:05:18 2022] POST /identity/v3/users => generated 160 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:18.454516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:18.473753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:18.474842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.475335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.475825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.476759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:18.477459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:18.480954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.498075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 499/984] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:18.503710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:18.574279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5a731-9ea9-43d9-8741-86713d169566 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:18.607527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b5495-e905-409c-97f2-26929e010ed4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:18.611135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:18.612047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.612430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.612802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.621301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:18.623280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:18.637550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:18.638798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 13 13:05:18.639810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:18.641477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 486/985] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:05:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:18.646804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:18.650193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:18.651318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.651782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.652226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.659733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:18.661410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:18.676133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:18.677037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:18.677971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:18.677971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:18.677971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:18.677971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:18.677971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:18.677971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:18.677971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:18.677971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:18.681002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 500/986] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:18.686441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:18.758253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11106185-5b34-41e5-a248-e566f3b2750c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:18.790419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fde79-0a67-4b15-bf03-4b8560d18b13 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:18.793538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:18.794127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.794483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.794830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.795713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:18.796090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:18.797625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.827856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666a13c-9455-4754-8775-d5587503cd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:18.828877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a13c-9455-4754-8775-d5587503cd41 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:18.829396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a13c-9455-4754-8775-d5587503cd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:18.829987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a13c-9455-4754-8775-d5587503cd41 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:18.831060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666a13c-9455-4754-8775-d5587503cd41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:18.831724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666a13c-9455-4754-8775-d5587503cd41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:18.833379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666a13c-9455-4754-8775-d5587503cd41 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:18.846186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 487/987] 10.222.0.25 () {56 vars in 1101 bytes} [Fri May 13 13:05:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:18.847517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 501/988] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:05:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:18.857844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:18.966156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e44f77-c94b-4dda-a68c-6e5f1b6e8bb5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:19.004641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50888484-b09c-4f35-8456-768e544c5170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:19.005320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50888484-b09c-4f35-8456-768e544c5170 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:19.005600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50888484-b09c-4f35-8456-768e544c5170 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:19.005906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50888484-b09c-4f35-8456-768e544c5170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:19.008895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50888484-b09c-4f35-8456-768e544c5170 None admin] RBAC: Adding query filter params (role.id=4db664dca3c640eea82b75a3e905becd, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=63d11bc6b12949f28957146b7d66a71d) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:19.009231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50888484-b09c-4f35-8456-768e544c5170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:19.010751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50888484-b09c-4f35-8456-768e544c5170 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:19.022013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 488/989] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:18 2022] GET /identity/v3/role_assignments?user.id=63d11bc6b12949f28957146b7d66a71d&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=4db664dca3c640eea82b75a3e905becd => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:19.856526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7fb4f-2d41-43b0-a9e7-2e128bfc30e8 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:19.857021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7fb4f-2d41-43b0-a9e7-2e128bfc30e8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:19.857021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7fb4f-2d41-43b0-a9e7-2e128bfc30e8 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:19.858446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 502/990] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:19.863794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d9b4-bfe6-4e01-933b-df65d3979455 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:19.864212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d9b4-bfe6-4e01-933b-df65d3979455 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:19.864382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d9b4-bfe6-4e01-933b-df65d3979455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:19.882456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5733d9b4-bfe6-4e01-933b-df65d3979455 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:19.884542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5733d9b4-bfe6-4e01-933b-df65d3979455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:19.950057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 489/991] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:19.972398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7fb4f-2d41-43b0-a9e7-2e128bfc30e8 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:20.068809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7fb4f-2d41-43b0-a9e7-2e128bfc30e8 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:20.100302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:20.101111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.101390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.101668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.102399 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:20.103780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:20.115755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:20.116850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:20.118283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 503/992] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:20.125409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d9b4-bfe6-4e01-933b-df65d3979455 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:20.224867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d9b4-bfe6-4e01-933b-df65d3979455 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:20.260080 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ecd02-e810-4bc6-8736-068af6ac51f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:20.260750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ecd02-e810-4bc6-8736-068af6ac51f9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.261038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ecd02-e810-4bc6-8736-068af6ac51f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.261315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ecd02-e810-4bc6-8736-068af6ac51f9 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.262053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ecd02-e810-4bc6-8736-068af6ac51f9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:20.262362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ecd02-e810-4bc6-8736-068af6ac51f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:20.263753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ecd02-e810-4bc6-8736-068af6ac51f9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:20.268644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 490/993] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:05:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:20.274416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:20.309951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c8259-6796-4355-9e41-e5dbb6026b2b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.310524 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c8259-6796-4355-9e41-e5dbb6026b2b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.310809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c8259-6796-4355-9e41-e5dbb6026b2b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.313171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 491/994] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:20.318010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47e3d3-b527-40da-b45a-f41795e21580 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.318299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47e3d3-b527-40da-b45a-f41795e21580 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.318890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47e3d3-b527-40da-b45a-f41795e21580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.335600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0e47e3d3-b527-40da-b45a-f41795e21580 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:20.336822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0e47e3d3-b527-40da-b45a-f41795e21580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:20.379858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88386b6-c382-4e23-99b9-a5d0f20e2bf2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:20.403912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 492/995] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:20.411661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:20.412325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.412613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.412893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.413601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:20.413945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:20.415847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:20.421824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 504/996] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:05:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:20.426292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47e3d3-b527-40da-b45a-f41795e21580 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:20.426440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:20.532161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47e3d3-b527-40da-b45a-f41795e21580 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:20.538094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f27fb-3955-4f31-aa73-8fc308f96298 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:20.568467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:20.569126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.569424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.569680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.570530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] RBAC: Adding query filter params (interface=public, service_id=9158631de5bd4f48bac59d0912e7ec80, region_id=RegionOne) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:20.570862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:20.572274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:20.573777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:20.575173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.575888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.576412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.577213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:20.577332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 493/997] 10.222.0.25 () {56 vars in 1251 bytes} [Fri May 13 13:05:20 2022] GET /identity/v3/endpoints?service_id=9158631de5bd4f48bac59d0912e7ec80&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:20.578981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:20.582559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:20.593440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:20.594674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 13 13:05:20.595704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:20.597329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 505/998] 10.222.0.25 () {56 vars in 1126 bytes} [Fri May 13 13:05:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:20.604822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:20.687692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793c2a7-8411-4c49-b23f-c1e9897f1ceb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:20.702712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484029fe-baec-4264-a74e-18c540c773d6 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:20.722748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69a37c-1f7a-4604-9636-bf9dff10aa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:20.723414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69a37c-1f7a-4604-9636-bf9dff10aa73 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.723696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69a37c-1f7a-4604-9636-bf9dff10aa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.723968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69a37c-1f7a-4604-9636-bf9dff10aa73 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.724639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69a37c-1f7a-4604-9636-bf9dff10aa73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:20.726046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69a37c-1f7a-4604-9636-bf9dff10aa73 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:20.741889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 494/999] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:05:20 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:05:20.746419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:20.747221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.747574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.747923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.748746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:20.749123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:20.750853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:20.756703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 506/1000] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:20.762347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69a37c-1f7a-4604-9636-bf9dff10aa73 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:20.866506 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69a37c-1f7a-4604-9636-bf9dff10aa73 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:20.901467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb99b03-fc4c-483a-ac4a-dde8d64884d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:20.902173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb99b03-fc4c-483a-ac4a-dde8d64884d8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:20.902453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb99b03-fc4c-483a-ac4a-dde8d64884d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:20.902725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb99b03-fc4c-483a-ac4a-dde8d64884d8 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:20.903433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb99b03-fc4c-483a-ac4a-dde8d64884d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:20.903733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb99b03-fc4c-483a-ac4a-dde8d64884d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:20.905113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb99b03-fc4c-483a-ac4a-dde8d64884d8 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:20.910486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 495/1001] 10.222.0.25 () {56 vars in 1122 bytes} [Fri May 13 13:05:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:20.916958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:21.015711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f7036-ab73-4e26-9498-ec0bd102b4ec None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:21.047055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b68a-2cd7-4ccc-ade1-5648d9923227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:21.047711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b68a-2cd7-4ccc-ade1-5648d9923227 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:21.047990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b68a-2cd7-4ccc-ade1-5648d9923227 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:21.048263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b68a-2cd7-4ccc-ade1-5648d9923227 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:21.049001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b68a-2cd7-4ccc-ade1-5648d9923227 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:21.050402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b68a-2cd7-4ccc-ade1-5648d9923227 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:21.165240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 507/1002] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:05:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:05:21.295870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714170f-787d-4337-bb83-1751a54de759 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:21.296057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714170f-787d-4337-bb83-1751a54de759 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:21.296347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714170f-787d-4337-bb83-1751a54de759 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:21.297972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 496/1003] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:21.303538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbeb2e-e995-40e8-b7e2-0195bd2d0cec None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:21.303850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbeb2e-e995-40e8-b7e2-0195bd2d0cec None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:21.304117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbeb2e-e995-40e8-b7e2-0195bd2d0cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:21.322572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6adbeb2e-e995-40e8-b7e2-0195bd2d0cec None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:21.323824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6adbeb2e-e995-40e8-b7e2-0195bd2d0cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:21.396662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 508/1004] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:21.418998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714170f-787d-4337-bb83-1751a54de759 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:21.517428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714170f-787d-4337-bb83-1751a54de759 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:21.550676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:21.552015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:21.552497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:21.552976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] PATH_INFO: `/v3/services/image` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:21.553832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:21.555381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:21.567326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:21.568331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:21.569163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:21.571316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 497/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Fri May 13 13:05:21 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:21.577374 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbeb2e-e995-40e8-b7e2-0195bd2d0cec None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:21.673610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbeb2e-e995-40e8-b7e2-0195bd2d0cec None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:21.707462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fd049-8522-47f7-a9ef-06bf441b29b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:21.707462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fd049-8522-47f7-a9ef-06bf441b29b8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:21.707462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fd049-8522-47f7-a9ef-06bf441b29b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:21.707462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fd049-8522-47f7-a9ef-06bf441b29b8 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:21.707877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fd049-8522-47f7-a9ef-06bf441b29b8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:21.708162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fd049-8522-47f7-a9ef-06bf441b29b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:21.709527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fd049-8522-47f7-a9ef-06bf441b29b8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:21.716375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 509/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:21.721779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:21.817252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b79af3-8f90-4d03-af45-e228299a7f10 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:21.848566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c041f6e-54c9-48cb-8463-8db5836d7a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:21.849227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c041f6e-54c9-48cb-8463-8db5836d7a56 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:21.849504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c041f6e-54c9-48cb-8463-8db5836d7a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:21.849776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c041f6e-54c9-48cb-8463-8db5836d7a56 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:21.850494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c041f6e-54c9-48cb-8463-8db5836d7a56 None admin] RBAC: Adding query filter params (type=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:21.850796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c041f6e-54c9-48cb-8463-8db5836d7a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:21.852671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c041f6e-54c9-48cb-8463-8db5836d7a56 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:21.857216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 498/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:22.993690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df41d-9089-43cb-a1db-e6b9b7b21a8a None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:22.994254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df41d-9089-43cb-a1db-e6b9b7b21a8a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:22.994333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068df41d-9089-43cb-a1db-e6b9b7b21a8a None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:22.997877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 510/1008] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:23.000902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596a42e-47af-4c78-afcc-1a8ff5ffade0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.001335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596a42e-47af-4c78-afcc-1a8ff5ffade0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.001500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596a42e-47af-4c78-afcc-1a8ff5ffade0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.017869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6596a42e-47af-4c78-afcc-1a8ff5ffade0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:23.019102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6596a42e-47af-4c78-afcc-1a8ff5ffade0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:23.084671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 499/1009] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:23.107897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df41d-9089-43cb-a1db-e6b9b7b21a8a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:23.203816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068df41d-9089-43cb-a1db-e6b9b7b21a8a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:23.238361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:23.239184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.239468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.239745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] PATH_INFO: `/v3/services/compute` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.240413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:23.241859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:23.254356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:23.255456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 13 13:05:23.256400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:23.263830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 511/1010] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:23.263976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596a42e-47af-4c78-afcc-1a8ff5ffade0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:23.356947 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596a42e-47af-4c78-afcc-1a8ff5ffade0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:23.389502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24240ad5-59e8-4ee7-b1fc-c082d18a271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:23.390205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24240ad5-59e8-4ee7-b1fc-c082d18a271a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.390492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24240ad5-59e8-4ee7-b1fc-c082d18a271a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.390770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24240ad5-59e8-4ee7-b1fc-c082d18a271a None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.391478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24240ad5-59e8-4ee7-b1fc-c082d18a271a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:23.391784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24240ad5-59e8-4ee7-b1fc-c082d18a271a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:23.393157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24240ad5-59e8-4ee7-b1fc-c082d18a271a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:23.403227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 500/1011] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:05:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:23.404416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:23.417739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746b53-9181-4b8b-bc9c-9e78c5ac0e6c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.418074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746b53-9181-4b8b-bc9c-9e78c5ac0e6c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.418326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746b53-9181-4b8b-bc9c-9e78c5ac0e6c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.419815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 501/1012] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:23.424554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b07ab-071b-4142-a1ce-53cf99f1690b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.424840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b07ab-071b-4142-a1ce-53cf99f1690b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.425119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b07ab-071b-4142-a1ce-53cf99f1690b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.441342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8b6b07ab-071b-4142-a1ce-53cf99f1690b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:23.442544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8b6b07ab-071b-4142-a1ce-53cf99f1690b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:23.509283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2fb9-4cb4-48bf-9c7c-2c76fa3ca9ec None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:23.512076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 502/1013] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:23.535371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b07ab-071b-4142-a1ce-53cf99f1690b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:23.544017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:23.544684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.544966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.545241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.545976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:23.546295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:23.547644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:23.557173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 512/1014] 10.222.0.25 () {56 vars in 1112 bytes} [Fri May 13 13:05:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:23.557926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:23.632878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b07ab-071b-4142-a1ce-53cf99f1690b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:23.667474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f4492-1d13-4642-9080-7e3aadb322f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:23.668294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f4492-1d13-4642-9080-7e3aadb322f3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.668574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f4492-1d13-4642-9080-7e3aadb322f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.668845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f4492-1d13-4642-9080-7e3aadb322f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.669233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10649cd6-9743-4a7e-b453-720c968364f5 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:23.670048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f4492-1d13-4642-9080-7e3aadb322f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:23.671492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f4492-1d13-4642-9080-7e3aadb322f3 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:23.673199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 503/1015] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:23.678698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f4492-1d13-4642-9080-7e3aadb322f3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:23.704719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6045e4-1f76-4fbf-a6d6-7070dadf2014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:23.705614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6045e4-1f76-4fbf-a6d6-7070dadf2014 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.706082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6045e4-1f76-4fbf-a6d6-7070dadf2014 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.706533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6045e4-1f76-4fbf-a6d6-7070dadf2014 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.707420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6045e4-1f76-4fbf-a6d6-7070dadf2014 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:23.708935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6045e4-1f76-4fbf-a6d6-7070dadf2014 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:23.775802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f4492-1d13-4642-9080-7e3aadb322f3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:23.811099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17db7a-20d4-4edf-9bcb-02146542b68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:23.817529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17db7a-20d4-4edf-9bcb-02146542b68b None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:23.817529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17db7a-20d4-4edf-9bcb-02146542b68b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:23.817529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17db7a-20d4-4edf-9bcb-02146542b68b None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:23.817529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17db7a-20d4-4edf-9bcb-02146542b68b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:23.817529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17db7a-20d4-4edf-9bcb-02146542b68b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:23.817917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 513/1016] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:05:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:05:23.954849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 504/1017] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:05:23 2022] POST /identity/v3/projects => generated 311 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:05:24.094112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6249a4-4a3b-4706-b121-5b80760f71a7 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:24.094623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6249a4-4a3b-4706-b121-5b80760f71a7 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:24.094623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6249a4-4a3b-4706-b121-5b80760f71a7 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:24.096188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 514/1018] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:24.102436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467e4ca-2c28-481a-85ec-00b1508d8a31 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:24.102964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467e4ca-2c28-481a-85ec-00b1508d8a31 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:24.103050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467e4ca-2c28-481a-85ec-00b1508d8a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:24.121436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b467e4ca-2c28-481a-85ec-00b1508d8a31 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:24.122717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b467e4ca-2c28-481a-85ec-00b1508d8a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:24.198530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 505/1019] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:24 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:24.220651 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6249a4-4a3b-4706-b121-5b80760f71a7 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:24.317616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6249a4-4a3b-4706-b121-5b80760f71a7 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:24.349198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abbe98-4dbb-4325-b651-95669766927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:24.349915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abbe98-4dbb-4325-b651-95669766927d None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:24.350201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abbe98-4dbb-4325-b651-95669766927d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:24.350477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abbe98-4dbb-4325-b651-95669766927d None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:24.351217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abbe98-4dbb-4325-b651-95669766927d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:24.353106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abbe98-4dbb-4325-b651-95669766927d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:24.548669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 515/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:05:24 2022] POST /identity/v3/services => generated 242 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:26.180405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ac371-bc12-4f38-8762-835cc1a9ae7c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.180976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ac371-bc12-4f38-8762-835cc1a9ae7c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.180976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ac371-bc12-4f38-8762-835cc1a9ae7c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.182510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 506/1021] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:26.187930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a6334-3276-4f48-8814-92bad6df6d27 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.188326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a6334-3276-4f48-8814-92bad6df6d27 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.188505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a6334-3276-4f48-8814-92bad6df6d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.204682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f08a6334-3276-4f48-8814-92bad6df6d27 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:26.205910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f08a6334-3276-4f48-8814-92bad6df6d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:26.265911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a40b3-3383-4432-a37e-b7c3946d0917 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.266198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a40b3-3383-4432-a37e-b7c3946d0917 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.266476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a40b3-3383-4432-a37e-b7c3946d0917 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.268038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 507/1022] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:26.272793 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07c001-5360-4263-b963-47aee853aba3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.273080 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07c001-5360-4263-b963-47aee853aba3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.273357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07c001-5360-4263-b963-47aee853aba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.276864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 516/1023] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:26.291570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ec07c001-5360-4263-b963-47aee853aba3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:26.292763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ec07c001-5360-4263-b963-47aee853aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:26.300277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a6334-3276-4f48-8814-92bad6df6d27 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:26.360265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 508/1024] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:26.384219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07c001-5360-4263-b963-47aee853aba3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:26.395656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a6334-3276-4f48-8814-92bad6df6d27 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:26.428875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558ade-7a9b-465a-ba63-6fe16d827f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:26.429727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558ade-7a9b-465a-ba63-6fe16d827f4d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.430057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558ade-7a9b-465a-ba63-6fe16d827f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.430339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558ade-7a9b-465a-ba63-6fe16d827f4d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.440637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67558ade-7a9b-465a-ba63-6fe16d827f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-67558ade-7a9b-465a-ba63-6fe16d827f4d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:26.449318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:26.450471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:26.451328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 517/1025] 10.222.0.25 () {56 vars in 1120 bytes} [Fri May 13 13:05:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:26.456423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558ade-7a9b-465a-ba63-6fe16d827f4d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:26.486079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07c001-5360-4263-b963-47aee853aba3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:26.520546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12775825-ddad-4df2-9a96-614e1204468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:26.521342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12775825-ddad-4df2-9a96-614e1204468a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.521629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12775825-ddad-4df2-9a96-614e1204468a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.521952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12775825-ddad-4df2-9a96-614e1204468a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.523135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12775825-ddad-4df2-9a96-614e1204468a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:26.524613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12775825-ddad-4df2-9a96-614e1204468a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:26.526597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 509/1026] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:26.531582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12775825-ddad-4df2-9a96-614e1204468a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:26.568213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558ade-7a9b-465a-ba63-6fe16d827f4d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:26.603869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:26.604570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.604859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.605137 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.605956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:26.606269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:26.607710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:26.616493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 518/1027] 10.222.0.25 () {56 vars in 1115 bytes} [Fri May 13 13:05:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:26.623105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:26.642007 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12775825-ddad-4df2-9a96-614e1204468a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:26.676301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce649-1ee0-40c1-974d-33b324fe1b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:26.676978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce649-1ee0-40c1-974d-33b324fe1b3a None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.677266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce649-1ee0-40c1-974d-33b324fe1b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.677551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce649-1ee0-40c1-974d-33b324fe1b3a None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.678318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ce649-1ee0-40c1-974d-33b324fe1b3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:26.679758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ce649-1ee0-40c1-974d-33b324fe1b3a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:26.719466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95186c43-b493-424d-91d0-5240b4d0250a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:26.751385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115f24b-9db8-475b-8612-bb1e20458e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:26.752185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115f24b-9db8-475b-8612-bb1e20458e62 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.752470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115f24b-9db8-475b-8612-bb1e20458e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.752746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115f24b-9db8-475b-8612-bb1e20458e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.764357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115f24b-9db8-475b-8612-bb1e20458e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:26.765815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115f24b-9db8-475b-8612-bb1e20458e62 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-7115f24b-9db8-475b-8612-bb1e20458e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:26.782278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:26.783199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:26.784867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:26.784867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:26.784867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:26.784867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:26.784867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 519/1028] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:26.789199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583ae04-534e-4b38-b36a-7358eaba246f None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.791867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583ae04-534e-4b38-b36a-7358eaba246f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.791867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583ae04-534e-4b38-b36a-7358eaba246f None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.792073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 510/1029] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:05:26 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:05:26.792282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 520/1030] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:26.792372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce649-1ee0-40c1-974d-33b324fe1b3a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:26.796344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da9a6b-90c7-47e0-a38c-aae2c9af2654 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.796638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da9a6b-90c7-47e0-a38c-aae2c9af2654 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.796918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da9a6b-90c7-47e0-a38c-aae2c9af2654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.815976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-56da9a6b-90c7-47e0-a38c-aae2c9af2654 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:26.817212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-56da9a6b-90c7-47e0-a38c-aae2c9af2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:26.889349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 521/1031] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:26.898659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce649-1ee0-40c1-974d-33b324fe1b3a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:26.912128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da9a6b-90c7-47e0-a38c-aae2c9af2654 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:26.934974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:26.935659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:26.935948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:26.936228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:26.936976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:26.937284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:26.938744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:26.968146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 511/1032] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:26.974181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:27.012968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da9a6b-90c7-47e0-a38c-aae2c9af2654 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.050317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.051402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.051669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.051943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] PATH_INFO: `/v3/services/image` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.052604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.054261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:27.067685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:27.069864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:27.070971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 522/1033] 10.222.0.25 () {56 vars in 1105 bytes} [Fri May 13 13:05:26 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:27.074979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:27.078025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffbf68-f9fc-4e6b-b5ab-20039416577d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.111088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.111915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.112199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.112477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.120199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.121641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:27.136056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:27.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:05:27.138364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:27.139049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 512/1034] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:05:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:27.142730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:27.174316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e24c2-0798-44ca-bd80-d1f47bc16c72 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.208733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.209409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.209670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.209987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.210814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] RBAC: Adding query filter params (name=image) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:27.210997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.213000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.240043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550f5b8-1656-4132-904a-df9cf436b9ec None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.272916 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.278825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.278825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.278825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.278825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:27.278825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.278825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.293979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 513/1035] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:05:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:27.299476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:27.320285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 523/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:27.326519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:27.401230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc9fd6-ae86-479f-8e9b-bbe55d2e7d34 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.425656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24317-88f0-46ad-9ecb-55f70defef02 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.434828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65488f82-e281-4618-bf8a-211d3ccac23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.435485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488f82-e281-4618-bf8a-211d3ccac23a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.435768 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488f82-e281-4618-bf8a-211d3ccac23a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.436046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488f82-e281-4618-bf8a-211d3ccac23a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.460528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.461223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.461513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.461816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.462531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:27.462842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.464216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.469262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 524/1037] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:27.476149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:27.514228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65488f82-e281-4618-bf8a-211d3ccac23a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.516068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65488f82-e281-4618-bf8a-211d3ccac23a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.672809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf171a97-cfaa-4eb9-93a9-1f766d551af8 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.706939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.707599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.708072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 514/1038] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:05:27.709946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.710267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.711084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] RBAC: Adding query filter params (interface=public, service_id=28df021b2b82495aba6d04c3d69b523a, region_id=RegionOne) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:27.711438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.713099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.713642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65488f82-e281-4618-bf8a-211d3ccac23a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:27.719243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 525/1039] 10.222.0.25 () {56 vars in 1251 bytes} [Fri May 13 13:05:27 2022] GET /identity/v3/endpoints?service_id=28df021b2b82495aba6d04c3d69b523a&interface=public®ion_id=RegionOne => generated 194 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:27.724526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:27.819371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65488f82-e281-4618-bf8a-211d3ccac23a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.824155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c26c1-1840-4ff4-9c3d-5c690eebf1a1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:27.856097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.856895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.857182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.857461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.867320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.869338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.870031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf88e1a-0e34-49b5-af64-4a7a3b2aaef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:27.870992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf88e1a-0e34-49b5-af64-4a7a3b2aaef1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:27.871424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf88e1a-0e34-49b5-af64-4a7a3b2aaef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:27.872142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf88e1a-0e34-49b5-af64-4a7a3b2aaef1 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:27.872970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf88e1a-0e34-49b5-af64-4a7a3b2aaef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:27.874501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf88e1a-0e34-49b5-af64-4a7a3b2aaef1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:27.882283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 526/1040] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:05:27 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:27.886834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:27.887896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:27.890706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:27.890706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:27.890706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:27.890706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:27.890706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 515/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:27.894039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf88e1a-0e34-49b5-af64-4a7a3b2aaef1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:28.005120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf88e1a-0e34-49b5-af64-4a7a3b2aaef1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:28.038811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:28.040022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:28.040310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:28.040583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:28.041313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:28.041619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:28.043048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:28.056730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 527/1042] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:28.062298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:28.160733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66add8-6356-4970-99e8-a3ea384309b6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:28.193549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:28.194643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:28.195070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:28.195535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:28.202184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:28.203801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:28.217661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:28.218717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:28.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:28.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:28.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:28.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:28.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:28.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:28.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:28.219895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:28.222002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 516/1043] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:28.228128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:28.320932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e64d65-d51f-4ce1-9c52-45f0730ea0d2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:28.353073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe1fe4-92cc-4a2f-b5f7-ec759dfbaf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:28.353731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe1fe4-92cc-4a2f-b5f7-ec759dfbaf17 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:28.354036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe1fe4-92cc-4a2f-b5f7-ec759dfbaf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:28.354305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe1fe4-92cc-4a2f-b5f7-ec759dfbaf17 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:28.355079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe1fe4-92cc-4a2f-b5f7-ec759dfbaf17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:28.355382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe1fe4-92cc-4a2f-b5f7-ec759dfbaf17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:28.356800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe1fe4-92cc-4a2f-b5f7-ec759dfbaf17 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:28.367894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 528/1044] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:05:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:28.373532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:28.467680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bce45-bcd1-4864-a3dd-926f5b2f1598 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:28.504669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfa250-f420-47af-869c-8bb56e5a3275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:28.505329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfa250-f420-47af-869c-8bb56e5a3275 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:28.505610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfa250-f420-47af-869c-8bb56e5a3275 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:28.505928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfa250-f420-47af-869c-8bb56e5a3275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:28.506758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfa250-f420-47af-869c-8bb56e5a3275 None admin] RBAC: Adding query filter params (role.id=4db664dca3c640eea82b75a3e905becd, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=3f94e52e32324d8cbf38fb05dc38ea96) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:28.507067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfa250-f420-47af-869c-8bb56e5a3275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:28.509048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfa250-f420-47af-869c-8bb56e5a3275 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:28.522052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 517/1045] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:28 2022] GET /identity/v3/role_assignments?user.id=3f94e52e32324d8cbf38fb05dc38ea96&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=4db664dca3c640eea82b75a3e905becd => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:29.105114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcada4b-1399-4b1c-ad29-01141708782e None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:29.105492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcada4b-1399-4b1c-ad29-01141708782e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:29.105695 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcada4b-1399-4b1c-ad29-01141708782e None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:29.107221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 529/1046] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:29.112521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2077b8-f1e2-4e7d-ad75-f761fdc76ead None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:29.112889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2077b8-f1e2-4e7d-ad75-f761fdc76ead None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:29.113274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2077b8-f1e2-4e7d-ad75-f761fdc76ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:29.129591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fe2077b8-f1e2-4e7d-ad75-f761fdc76ead None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:29.130847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fe2077b8-f1e2-4e7d-ad75-f761fdc76ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:29.206110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 518/1047] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:29.231142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcada4b-1399-4b1c-ad29-01141708782e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:29.327213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcada4b-1399-4b1c-ad29-01141708782e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:29.358642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f6f8-1bb5-4361-912d-027f34b25841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:29.359368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f6f8-1bb5-4361-912d-027f34b25841 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:29.359653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f6f8-1bb5-4361-912d-027f34b25841 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:29.359927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f6f8-1bb5-4361-912d-027f34b25841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:29.370432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2f6f8-1bb5-4361-912d-027f34b25841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-38b2f6f8-1bb5-4361-912d-027f34b25841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:29.378021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:29.378923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:05:29.379773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:29.382140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 530/1048] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:05:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:29.385062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2077b8-f1e2-4e7d-ad75-f761fdc76ead None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:29.482731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2077b8-f1e2-4e7d-ad75-f761fdc76ead None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:29.518187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:29.518861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:29.519144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:29.519478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:29.520262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:29.520572 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:29.521967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:29.534431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 519/1049] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:05:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:29.535583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f6f8-1bb5-4361-912d-027f34b25841 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:29.636644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f6f8-1bb5-4361-912d-027f34b25841 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:29.670916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ba2c-29df-43dd-b66f-71b0a2c39131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:29.672105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ba2c-29df-43dd-b66f-71b0a2c39131 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:29.672391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ba2c-29df-43dd-b66f-71b0a2c39131 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:29.672667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ba2c-29df-43dd-b66f-71b0a2c39131 None admin] PATH_INFO: `/v3/users/74b3416e7d3641088d86e1227c9d4716` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:29.685830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239ba2c-29df-43dd-b66f-71b0a2c39131 None admin] RBAC: Authorizing `identity:get_user(user_id=74b3416e7d3641088d86e1227c9d4716)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:29.687290 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239ba2c-29df-43dd-b66f-71b0a2c39131 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:29.693430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 531/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:29 2022] GET /identity/v3/users/74b3416e7d3641088d86e1227c9d4716 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:29.694657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:29.793218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb765a-7337-431b-8bf1-4713ac614eb2 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:29.824956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7b21a-94eb-41e3-acfb-b497c803177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:29.825608 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7b21a-94eb-41e3-acfb-b497c803177c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:29.825922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7b21a-94eb-41e3-acfb-b497c803177c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:29.826206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7b21a-94eb-41e3-acfb-b497c803177c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:29.889596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7b21a-94eb-41e3-acfb-b497c803177c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:29.891143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7b21a-94eb-41e3-acfb-b497c803177c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:29.993937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 520/1051] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:05:29.995660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ba2c-29df-43dd-b66f-71b0a2c39131 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:30.092185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ba2c-29df-43dd-b66f-71b0a2c39131 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:30.125149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2fd27-e675-4ccf-839c-74de5f0eec39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:30.126036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2fd27-e675-4ccf-839c-74de5f0eec39 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.126321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2fd27-e675-4ccf-839c-74de5f0eec39 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.126595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2fd27-e675-4ccf-839c-74de5f0eec39 None admin] PATH_INFO: `/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.127775 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2fd27-e675-4ccf-839c-74de5f0eec39 None admin] RBAC: Authorizing `identity:get_project(project_id=2453833aba8d45fb8c4c4ee1a28b4cec)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:30.129212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2fd27-e675-4ccf-839c-74de5f0eec39 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:30.135981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 532/1052] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:05:29 2022] GET /identity/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:30.137193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7b21a-94eb-41e3-acfb-b497c803177c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:30.143579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea4ef-f113-4bf2-9dd1-1d268b840e26 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.143811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea4ef-f113-4bf2-9dd1-1d268b840e26 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.144093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea4ef-f113-4bf2-9dd1-1d268b840e26 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.145431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 533/1053] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:30.150954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6d975-d4a4-4549-8b0e-e8ed9c29829c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.151246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6d975-d4a4-4549-8b0e-e8ed9c29829c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.151526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6d975-d4a4-4549-8b0e-e8ed9c29829c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.168085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bac6d975-d4a4-4549-8b0e-e8ed9c29829c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:30.169261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bac6d975-d4a4-4549-8b0e-e8ed9c29829c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:30.237725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7b21a-94eb-41e3-acfb-b497c803177c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:30.238961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 534/1054] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:30.261623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6d975-d4a4-4549-8b0e-e8ed9c29829c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:30.271219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:30.271881 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.272165 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.272547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.273271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=2453833aba8d45fb8c4c4ee1a28b4cec, user.id=74b3416e7d3641088d86e1227c9d4716) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:30.273613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:30.275074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:30.289815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 521/1055] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:30 2022] GET /identity/v3/role_assignments?user.id=74b3416e7d3641088d86e1227c9d4716&scope.project.id=2453833aba8d45fb8c4c4ee1a28b4cec&role.id=663341777b0b44128ed78fe8d40f6462 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:30.363145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6d975-d4a4-4549-8b0e-e8ed9c29829c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:30.398178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:30.399440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.399726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.399999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] PATH_INFO: `/v3/services/image` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.400663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:30.402083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:30.414263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:30.415826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 13 13:05:30.416697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:30.421826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 535/1056] 10.222.0.25 () {56 vars in 1105 bytes} [Fri May 13 13:05:30 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:30.422393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:30.526061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07388d-d00d-42f2-a49b-06b8ee4a3c51 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:30.561208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3017e03-116d-43c2-a2f2-d01bd06753a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:30.561902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017e03-116d-43c2-a2f2-d01bd06753a5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.562192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017e03-116d-43c2-a2f2-d01bd06753a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.562472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017e03-116d-43c2-a2f2-d01bd06753a5 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.563185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3017e03-116d-43c2-a2f2-d01bd06753a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:30.563495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3017e03-116d-43c2-a2f2-d01bd06753a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:30.564868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3017e03-116d-43c2-a2f2-d01bd06753a5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:30.574361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 522/1057] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:30.575398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:30.673504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bef75c-17fd-433b-9c48-a5d90699a10f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:30.701582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956628bf-21cc-46a5-9640-41c55e80b0c0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.701902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956628bf-21cc-46a5-9640-41c55e80b0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.702167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956628bf-21cc-46a5-9640-41c55e80b0c0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.705905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 523/1058] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:30.707953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:30.708400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238f288-92c2-4eec-87bf-83fecb754c3b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.708616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.708719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238f288-92c2-4eec-87bf-83fecb754c3b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.708902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.709003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238f288-92c2-4eec-87bf-83fecb754c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.709177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.709910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] RBAC: Adding query filter params (type=image) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:30.710287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:30.712109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:30.720749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 536/1059] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:30.723178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:30.725689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e238f288-92c2-4eec-87bf-83fecb754c3b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:30.727472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e238f288-92c2-4eec-87bf-83fecb754c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:30.794253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 524/1060] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:30.816439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238f288-92c2-4eec-87bf-83fecb754c3b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:30.820874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e084a-7e29-485c-a5f1-d9620e006e93 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:30.852701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df285d3-0570-4220-8a35-bb9c8da35519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:30.853356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df285d3-0570-4220-8a35-bb9c8da35519 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.853637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df285d3-0570-4220-8a35-bb9c8da35519 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.853956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df285d3-0570-4220-8a35-bb9c8da35519 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.854696 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df285d3-0570-4220-8a35-bb9c8da35519 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:30.856058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df285d3-0570-4220-8a35-bb9c8da35519 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:30.912017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238f288-92c2-4eec-87bf-83fecb754c3b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:30.933332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 537/1061] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:05:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:05:30.953320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b437dc-b4b5-41c4-b31b-2ed8c9323ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:30.954193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b437dc-b4b5-41c4-b31b-2ed8c9323ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:30.954480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b437dc-b4b5-41c4-b31b-2ed8c9323ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:30.954758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b437dc-b4b5-41c4-b31b-2ed8c9323ef3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:30.967464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b437dc-b4b5-41c4-b31b-2ed8c9323ef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-99b437dc-b4b5-41c4-b31b-2ed8c9323ef3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:30.976194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:30.977253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:30.978623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 525/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Fri May 13 13:05:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:30.983428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df285d3-0570-4220-8a35-bb9c8da35519 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:31.090022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df285d3-0570-4220-8a35-bb9c8da35519 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:31.123928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:31.124617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:31.124901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:31.125183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:31.125966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:31.126280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:31.127669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:31.136733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 538/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Fri May 13 13:05:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:31.142602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b437dc-b4b5-41c4-b31b-2ed8c9323ef3 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:31.240732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b437dc-b4b5-41c4-b31b-2ed8c9323ef3 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:31.273528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:31.274429 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:31.274718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:31.274997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:31.285220 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:31.286706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:31.303193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:31.304088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:31.305473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:31.305473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:31.305473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:31.305473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:31.305473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 526/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:31.310728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:31.406765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e2184-2532-49d6-bd2c-230720e4782e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:31.440999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:31.441668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:31.441982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:31.442259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:31.442976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:31.443281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:31.444701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:31.461052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 539/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:31.462708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:31.567932 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78924c-1ba7-4739-8e21-c59632398399 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:31.599134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:31.599920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:31.600202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:31.600479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:31.608230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:31.609667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:31.623771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:31.624735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:31.625692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 527/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:05:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:31.630595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:31.738830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413fba8-b7ac-4b9c-89e1-1b94b82a216f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:31.773727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:31.774798 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:31.775332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:31.775816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:31.776943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:31.777554 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:31.779289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:31.798840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 540/1067] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:05:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:31.810835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:31.911228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d3e71-258d-4ce0-b5c8-dbee4ea606a7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:31.946403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:31.947450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:31.947838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:31.948212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:31.959237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:31.960809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:31.977664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:31.978809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:31.980738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:31.980738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:31.980738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:31.980738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:31.981840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 528/1068] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:31.986704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:32.090519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362d177-781d-4ebd-8e6b-a03e91a22aa8 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:32.123684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:32.124505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:32.124912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:32.125288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:32.126143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:32.126555 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:32.128029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:32.144159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 541/1069] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:32.145350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:32.240822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186cdd1-cbb6-4ef4-815f-36f9e227addb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:32.273522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:32.274496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:32.274785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:32.275064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:32.281504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:32.282994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:32.297361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:32.298278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:32.303799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 529/1070] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:32.304286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:32.404026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f14031-53c5-42b5-a32d-2fbc9ac51b9d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:32.437814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299609d-76bb-48c5-902d-3b01e4a08895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:32.438486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299609d-76bb-48c5-902d-3b01e4a08895 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:32.438771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299609d-76bb-48c5-902d-3b01e4a08895 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:32.439051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299609d-76bb-48c5-902d-3b01e4a08895 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:32.439836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299609d-76bb-48c5-902d-3b01e4a08895 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:32.440147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299609d-76bb-48c5-902d-3b01e4a08895 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:32.441577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299609d-76bb-48c5-902d-3b01e4a08895 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:32.454365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 542/1071] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:05:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:32.459492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:32.522191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a1a3-ad13-4848-873f-69903a9dffdb None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:32.522781 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a1a3-ad13-4848-873f-69903a9dffdb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:32.523136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a1a3-ad13-4848-873f-69903a9dffdb None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:32.525089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 543/1072] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:32.529940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438110aa-c178-4771-bdf1-7ebc8637a0fe None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:32.530233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438110aa-c178-4771-bdf1-7ebc8637a0fe None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:32.530515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438110aa-c178-4771-bdf1-7ebc8637a0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:32.547261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-438110aa-c178-4771-bdf1-7ebc8637a0fe None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:32.548445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-438110aa-c178-4771-bdf1-7ebc8637a0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:32.563408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d2a2b-13dc-4674-ab68-742db8a758c0 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:32.596229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d222de-72b0-4e63-8413-585809912199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:32.597114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d222de-72b0-4e63-8413-585809912199 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:32.597396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d222de-72b0-4e63-8413-585809912199 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:32.597671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d222de-72b0-4e63-8413-585809912199 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/3f94e52e32324d8cbf38fb05dc38ea96/roles/4db664dca3c640eea82b75a3e905becd` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:32.599899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d222de-72b0-4e63-8413-585809912199 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=3f94e52e32324d8cbf38fb05dc38ea96, role_id=4db664dca3c640eea82b75a3e905becd)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:32.601347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d222de-72b0-4e63-8413-585809912199 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:32.617170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 544/1073] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:32.643888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438110aa-c178-4771-bdf1-7ebc8637a0fe None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:32.690361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 530/1074] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:05:32 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/3f94e52e32324d8cbf38fb05dc38ea96/roles/4db664dca3c640eea82b75a3e905becd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:05:32.747111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438110aa-c178-4771-bdf1-7ebc8637a0fe None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:32.785200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cb912-2f8d-4f55-b021-c2a297361cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:32.785966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cb912-2f8d-4f55-b021-c2a297361cdc None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:32.786255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cb912-2f8d-4f55-b021-c2a297361cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:32.786534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cb912-2f8d-4f55-b021-c2a297361cdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:32.798567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cb912-2f8d-4f55-b021-c2a297361cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-6a1cb912-2f8d-4f55-b021-c2a297361cdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:32.806376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:32.807414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:32.808529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 545/1075] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:05:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:32.815778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d222de-72b0-4e63-8413-585809912199 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:32.918335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d222de-72b0-4e63-8413-585809912199 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:32.950988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:32.951658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:32.951940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:32.952215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:32.952952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:32.953257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:32.954654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:32.963192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 531/1076] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:05:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:32.969766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cb912-2f8d-4f55-b021-c2a297361cdc None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:33.069404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cb912-2f8d-4f55-b021-c2a297361cdc None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:33.102316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1566-2307-4ee3-9ff2-73cd6661dc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:33.103225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1566-2307-4ee3-9ff2-73cd6661dc66 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:33.103512 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1566-2307-4ee3-9ff2-73cd6661dc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:33.103787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1566-2307-4ee3-9ff2-73cd6661dc66 None admin] PATH_INFO: `/v3/users/74b3416e7d3641088d86e1227c9d4716` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:33.104974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1566-2307-4ee3-9ff2-73cd6661dc66 None admin] RBAC: Authorizing `identity:get_user(user_id=74b3416e7d3641088d86e1227c9d4716)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:33.106450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1566-2307-4ee3-9ff2-73cd6661dc66 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:33.112528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 546/1077] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:32 2022] GET /identity/v3/users/74b3416e7d3641088d86e1227c9d4716 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:33.114084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:33.198043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f2bb-61b1-4bbb-b181-5c8f8d8abe29 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:33.198383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f2bb-61b1-4bbb-b181-5c8f8d8abe29 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:33.198613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08f2bb-61b1-4bbb-b181-5c8f8d8abe29 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:33.200140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 547/1078] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:33.204980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6e98f-c546-4f69-a075-06e019ff4469 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:33.205273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6e98f-c546-4f69-a075-06e019ff4469 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:33.205555 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6e98f-c546-4f69-a075-06e019ff4469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:33.210919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84158c45-73e6-411f-9580-3486e14cd8ab None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:33.221961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-80b6e98f-c546-4f69-a075-06e019ff4469 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:33.223143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-80b6e98f-c546-4f69-a075-06e019ff4469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:33.244077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1ed65-3006-42e7-ae2d-105bdec90f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:33.245279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1ed65-3006-42e7-ae2d-105bdec90f94 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:33.245778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1ed65-3006-42e7-ae2d-105bdec90f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:33.246287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1ed65-3006-42e7-ae2d-105bdec90f94 None admin] PATH_INFO: `/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:33.248307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1ed65-3006-42e7-ae2d-105bdec90f94 None admin] RBAC: Authorizing `identity:get_project(project_id=2453833aba8d45fb8c4c4ee1a28b4cec)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:33.250017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1ed65-3006-42e7-ae2d-105bdec90f94 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:33.251943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 532/1079] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:05:33 2022] GET /identity/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:33.257627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1ed65-3006-42e7-ae2d-105bdec90f94 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:33.288024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 548/1080] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:33.310577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6e98f-c546-4f69-a075-06e019ff4469 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:33.354348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1ed65-3006-42e7-ae2d-105bdec90f94 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:33.386663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea4848-c67c-4491-bf68-726d5714e3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:33.387551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea4848-c67c-4491-bf68-726d5714e3c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:33.387825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea4848-c67c-4491-bf68-726d5714e3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:33.388099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea4848-c67c-4491-bf68-726d5714e3c4 None admin] PATH_INFO: `/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec/users/74b3416e7d3641088d86e1227c9d4716/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:33.390295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea4848-c67c-4491-bf68-726d5714e3c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2453833aba8d45fb8c4c4ee1a28b4cec, user_id=74b3416e7d3641088d86e1227c9d4716, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:33.391722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea4848-c67c-4491-bf68-726d5714e3c4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:33.405196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6e98f-c546-4f69-a075-06e019ff4469 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:33.437691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f7bc2-1b5c-467d-8067-361d91802bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:33.438454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7bc2-1b5c-467d-8067-361d91802bfe None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:33.438737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7bc2-1b5c-467d-8067-361d91802bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:33.439013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7bc2-1b5c-467d-8067-361d91802bfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:33.485188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 533/1081] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:05:33 2022] PUT /identity/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec/users/74b3416e7d3641088d86e1227c9d4716/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:05:33.510298 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f7bc2-1b5c-467d-8067-361d91802bfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:33.511965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f7bc2-1b5c-467d-8067-361d91802bfe None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:33.615611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 549/1082] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 13 13:05:33.621802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea4848-c67c-4491-bf68-726d5714e3c4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:33.721732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea4848-c67c-4491-bf68-726d5714e3c4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:33.754214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b0ca9-d228-4ede-9af5-f5cdc77628eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:33.755010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b0ca9-d228-4ede-9af5-f5cdc77628eb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:33.755299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b0ca9-d228-4ede-9af5-f5cdc77628eb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:33.755579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b0ca9-d228-4ede-9af5-f5cdc77628eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:33.766407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b0ca9-d228-4ede-9af5-f5cdc77628eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:33.767859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b0ca9-d228-4ede-9af5-f5cdc77628eb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-188b0ca9-d228-4ede-9af5-f5cdc77628eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:33.784464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:33.785378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:33.786408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:33.786408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:33.786408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:33.786408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:33.786408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 534/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:33.793011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f7bc2-1b5c-467d-8067-361d91802bfe None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:33.893375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f7bc2-1b5c-467d-8067-361d91802bfe None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:33.926571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c8e90-f51e-4edc-8a6b-d89f2e4bbd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:33.927245 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c8e90-f51e-4edc-8a6b-d89f2e4bbd05 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:33.927530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c8e90-f51e-4edc-8a6b-d89f2e4bbd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:33.927808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c8e90-f51e-4edc-8a6b-d89f2e4bbd05 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:33.928528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c8e90-f51e-4edc-8a6b-d89f2e4bbd05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:33.928837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c8e90-f51e-4edc-8a6b-d89f2e4bbd05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:33.930428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c8e90-f51e-4edc-8a6b-d89f2e4bbd05 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:33.949831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 550/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:34.921327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4074-7a96-4b58-b1a7-8a9149ab184d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:34.921883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4074-7a96-4b58-b1a7-8a9149ab184d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:34.921883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4074-7a96-4b58-b1a7-8a9149ab184d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:34.923203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 535/1085] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:34.928510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ae125-1606-403e-9e91-eb5592a6885d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:34.928933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ae125-1606-403e-9e91-eb5592a6885d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:34.929089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ae125-1606-403e-9e91-eb5592a6885d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:34.945519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-352ae125-1606-403e-9e91-eb5592a6885d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:34.946745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-352ae125-1606-403e-9e91-eb5592a6885d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:35.009316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 551/1086] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:35.032955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d4074-7a96-4b58-b1a7-8a9149ab184d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:35.129173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d4074-7a96-4b58-b1a7-8a9149ab184d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:35.160435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59e4a8-06a1-4863-b506-49c14b652f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:35.161245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e4a8-06a1-4863-b506-49c14b652f23 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.161522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e4a8-06a1-4863-b506-49c14b652f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.161810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59e4a8-06a1-4863-b506-49c14b652f23 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.173069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59e4a8-06a1-4863-b506-49c14b652f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5d59e4a8-06a1-4863-b506-49c14b652f23 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:35.181125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:35.182101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:35.183212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 536/1087] 10.222.0.25 () {56 vars in 1120 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:35.188633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ae125-1606-403e-9e91-eb5592a6885d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:35.287623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ae125-1606-403e-9e91-eb5592a6885d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:35.320089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:35.321273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.321557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.321860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.322603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:35.322909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:35.324281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:35.331963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 552/1088] 10.222.0.25 () {56 vars in 1115 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:35.338193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59e4a8-06a1-4863-b506-49c14b652f23 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:35.430982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59e4a8-06a1-4863-b506-49c14b652f23 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:35.463708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:35.464491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.464775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.465049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.474713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:35.477210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:35.493385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:35.494487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:35.495513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:35.495513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:35.495513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:35.495513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:35.495513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 537/1089] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:35.501040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:35.605551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354794e-75b9-4bfc-a274-1f9acc9eadc6 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:35.637893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb8ff2-1d52-4752-8164-2dc381cce5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:35.638556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8ff2-1d52-4752-8164-2dc381cce5db None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.638840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8ff2-1d52-4752-8164-2dc381cce5db None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.639116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb8ff2-1d52-4752-8164-2dc381cce5db None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.639831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb8ff2-1d52-4752-8164-2dc381cce5db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:35.640139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb8ff2-1d52-4752-8164-2dc381cce5db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:35.641510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb8ff2-1d52-4752-8164-2dc381cce5db None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:35.654014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 553/1090] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:35.659502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:35.701476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf13a5-4409-4536-92ae-cec2ebf875ce None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.701687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf13a5-4409-4536-92ae-cec2ebf875ce None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.701995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf13a5-4409-4536-92ae-cec2ebf875ce None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.703330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 554/1091] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:05:35.708368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a7a5-c917-435c-9283-2e75536b7b6c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.708650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a7a5-c917-435c-9283-2e75536b7b6c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.708925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07a7a5-c917-435c-9283-2e75536b7b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.725006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-7c07a7a5-c917-435c-9283-2e75536b7b6c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:35.726219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7c07a7a5-c917-435c-9283-2e75536b7b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:35.755150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c3f1c-4c39-4b59-a024-0f896728498f None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:35.786534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:35.787317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.787609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.787805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 555/1092] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:35.787962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] PATH_INFO: `/v3/users/nova` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.795817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:35.797253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:35.810926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07a7a5-c917-435c-9283-2e75536b7b6c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:05:35.819024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:35.820266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 538/1093] 10.222.0.25 () {56 vars in 1093 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:35.821329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:35.903413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07a7a5-c917-435c-9283-2e75536b7b6c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:35.911112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d9d52-d6e4-4247-96bb-bc66055f83db None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:35.935763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a3754-66e8-4183-8d4c-7721bf43213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:35.936545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a3754-66e8-4183-8d4c-7721bf43213b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.936823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a3754-66e8-4183-8d4c-7721bf43213b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.937094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a3754-66e8-4183-8d4c-7721bf43213b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.942956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:35.943614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:35.943900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:35.944179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:35.945012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:35.945354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:35.946740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a3754-66e8-4183-8d4c-7721bf43213b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:35.946859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-f61a3754-66e8-4183-8d4c-7721bf43213b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:35.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:35.955131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:05:35.956021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 556/1094] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:35.961653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a3754-66e8-4183-8d4c-7721bf43213b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:35.970482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 539/1095] 10.222.0.25 () {56 vars in 1133 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:35.971441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.054523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a3754-66e8-4183-8d4c-7721bf43213b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.065003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f51ed-959e-4278-a079-6e216295f0b7 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.086862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.087508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.087787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.088060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.088788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:36.089091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.090488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.098220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 557/1096] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:36.098568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54997f42-5153-4de8-ab89-5daef63a7d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.099281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54997f42-5153-4de8-ab89-5daef63a7d07 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.099574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54997f42-5153-4de8-ab89-5daef63a7d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.099848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54997f42-5153-4de8-ab89-5daef63a7d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.104215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.163004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54997f42-5153-4de8-ab89-5daef63a7d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.165015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54997f42-5153-4de8-ab89-5daef63a7d07 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.196385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69315d-785f-464e-8b3a-b0928f46e759 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.228033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a702b-fd47-4771-b76b-960181b8af9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.228901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a702b-fd47-4771-b76b-960181b8af9a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.229183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a702b-fd47-4771-b76b-960181b8af9a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.229456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a702b-fd47-4771-b76b-960181b8af9a None admin] PATH_INFO: `/v3/users/74b3416e7d3641088d86e1227c9d4716` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.230693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a702b-fd47-4771-b76b-960181b8af9a None admin] RBAC: Authorizing `identity:get_user(user_id=74b3416e7d3641088d86e1227c9d4716)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.232119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a702b-fd47-4771-b76b-960181b8af9a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.238312 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 558/1097] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/users/74b3416e7d3641088d86e1227c9d4716 => generated 285 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:36.239356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a702b-fd47-4771-b76b-960181b8af9a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.257596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 540/1098] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:36.263101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54997f42-5153-4de8-ab89-5daef63a7d07 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.332388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a702b-fd47-4771-b76b-960181b8af9a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.354011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54997f42-5153-4de8-ab89-5daef63a7d07 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.363704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a443a-d217-44ed-b4c1-8efc4cbb6943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.364351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a443a-d217-44ed-b4c1-8efc4cbb6943 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.364636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a443a-d217-44ed-b4c1-8efc4cbb6943 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.364917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a443a-d217-44ed-b4c1-8efc4cbb6943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.385998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.386779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.387060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.387329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.396831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.398300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:05:36.415403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:36.416467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:36.417388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:05:36.417388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:05:36.417388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:05:36.417388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:36.417388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 541/1099] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:36.422041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.428121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a443a-d217-44ed-b4c1-8efc4cbb6943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.430073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a443a-d217-44ed-b4c1-8efc4cbb6943 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.514716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2857f0-a168-48c2-9e3f-8c0d591127c1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.522045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 559/1100] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:36.527731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a443a-d217-44ed-b4c1-8efc4cbb6943 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.546270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.546935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.547213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.547481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.548196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:36.548500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.549882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.561438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 542/1101] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:36.566859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.623313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a443a-d217-44ed-b4c1-8efc4cbb6943 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.654686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac7871-e8a1-4218-bd70-f92c60b065e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.655530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac7871-e8a1-4218-bd70-f92c60b065e4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.655811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac7871-e8a1-4218-bd70-f92c60b065e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.656082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac7871-e8a1-4218-bd70-f92c60b065e4 None admin] PATH_INFO: `/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.657246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac7871-e8a1-4218-bd70-f92c60b065e4 None admin] RBAC: Authorizing `identity:get_project(project_id=2453833aba8d45fb8c4c4ee1a28b4cec)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.658698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac7871-e8a1-4218-bd70-f92c60b065e4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.659465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210edef5-9be8-42e5-8037-b3aca7e3cd11 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.660355 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 560/1102] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:36.666207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac7871-e8a1-4218-bd70-f92c60b065e4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.691871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.692667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.692953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.693228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.701764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.703356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:05:36.718139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:36.719236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:36.720160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 543/1103] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:05:36.725199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.767627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac7871-e8a1-4218-bd70-f92c60b065e4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.799468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.800115 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.800402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.800676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.801492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=2453833aba8d45fb8c4c4ee1a28b4cec, user.id=74b3416e7d3641088d86e1227c9d4716) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:36.801812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.803260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.809665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 561/1104] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/role_assignments?user.id=74b3416e7d3641088d86e1227c9d4716&scope.project.id=2453833aba8d45fb8c4c4ee1a28b4cec&role.id=663341777b0b44128ed78fe8d40f6462 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:36.820775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845e631-9359-41fb-bd95-df38c879ad83 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:36.857779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2636-d154-4b9c-af9f-fd451d9d06c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:36.858487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2636-d154-4b9c-af9f-fd451d9d06c9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:36.858773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2636-d154-4b9c-af9f-fd451d9d06c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:36.859052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2636-d154-4b9c-af9f-fd451d9d06c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:36.859859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2636-d154-4b9c-af9f-fd451d9d06c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:36.860174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2636-d154-4b9c-af9f-fd451d9d06c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:36.861649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2636-d154-4b9c-af9f-fd451d9d06c9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:36.874777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 544/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:05:36.881484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:36.982372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e840-d5f2-4ff3-9d2d-0153c5091f52 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:37.016093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087361f-4512-4e02-a293-4581dda4c10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:37.016751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087361f-4512-4e02-a293-4581dda4c10f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:37.017032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087361f-4512-4e02-a293-4581dda4c10f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:37.017305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087361f-4512-4e02-a293-4581dda4c10f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:37.018222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087361f-4512-4e02-a293-4581dda4c10f None admin] RBAC: Adding query filter params (role.id=4db664dca3c640eea82b75a3e905becd, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=3f94e52e32324d8cbf38fb05dc38ea96) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:37.018469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087361f-4512-4e02-a293-4581dda4c10f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:37.019892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087361f-4512-4e02-a293-4581dda4c10f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:37.025661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 562/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:36 2022] GET /identity/v3/role_assignments?user.id=3f94e52e32324d8cbf38fb05dc38ea96&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=4db664dca3c640eea82b75a3e905becd => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:39.064757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d897cc5-948d-4e98-ac2f-a6fdb761db7e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:39.065316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d897cc5-948d-4e98-ac2f-a6fdb761db7e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:39.065316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d897cc5-948d-4e98-ac2f-a6fdb761db7e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:39.066708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 545/1107] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:39.072607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896edf4e-e6e5-4ee4-8c95-854f7f570b0a None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:39.073031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896edf4e-e6e5-4ee4-8c95-854f7f570b0a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:39.073148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896edf4e-e6e5-4ee4-8c95-854f7f570b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:39.089811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-896edf4e-e6e5-4ee4-8c95-854f7f570b0a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:39.091004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-896edf4e-e6e5-4ee4-8c95-854f7f570b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:39.151596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 563/1108] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:39.175852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d897cc5-948d-4e98-ac2f-a6fdb761db7e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:39.270954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d897cc5-948d-4e98-ac2f-a6fdb761db7e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:39.302449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbec90-2c86-43eb-9d5e-bfcca8e2beaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:39.303213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbec90-2c86-43eb-9d5e-bfcca8e2beaa None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:39.303571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbec90-2c86-43eb-9d5e-bfcca8e2beaa None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:39.303852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbec90-2c86-43eb-9d5e-bfcca8e2beaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:39.305559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbec90-2c86-43eb-9d5e-bfcca8e2beaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:39.307019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbec90-2c86-43eb-9d5e-bfcca8e2beaa None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:39.308852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 546/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:39.314404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896edf4e-e6e5-4ee4-8c95-854f7f570b0a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:39.405584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896edf4e-e6e5-4ee4-8c95-854f7f570b0a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:39.436875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bcd8c-fbfd-4e98-928b-a4a63729292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:39.437526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bcd8c-fbfd-4e98-928b-a4a63729292c None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:39.437822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bcd8c-fbfd-4e98-928b-a4a63729292c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:39.438099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bcd8c-fbfd-4e98-928b-a4a63729292c None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:39.438800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bcd8c-fbfd-4e98-928b-a4a63729292c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:39.440209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bcd8c-fbfd-4e98-928b-a4a63729292c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:39.592863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 564/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:05:39 2022] POST /identity/v3/users => generated 286 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:41.838940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231997-c5a3-4d6c-b7f5-a25b5747da25 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:41.839439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231997-c5a3-4d6c-b7f5-a25b5747da25 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:41.839439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231997-c5a3-4d6c-b7f5-a25b5747da25 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:41.840784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 547/1111] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:41.846094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf562-d723-440c-84e2-9ed1b9ca3006 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:41.846490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf562-d723-440c-84e2-9ed1b9ca3006 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:41.847644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf562-d723-440c-84e2-9ed1b9ca3006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:41.863615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bffcf562-d723-440c-84e2-9ed1b9ca3006 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:41.864886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bffcf562-d723-440c-84e2-9ed1b9ca3006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:41.927363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 565/1112] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:41.951685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29231997-c5a3-4d6c-b7f5-a25b5747da25 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:42.048041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29231997-c5a3-4d6c-b7f5-a25b5747da25 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:42.080039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1ac0-23e6-4758-9589-f09970f123ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:42.080883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1ac0-23e6-4758-9589-f09970f123ac None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:42.081169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1ac0-23e6-4758-9589-f09970f123ac None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:42.081446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1ac0-23e6-4758-9589-f09970f123ac None admin] PATH_INFO: `/v3/roles/9065c36fc305418baa5b977858b445dd` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:42.082938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1ac0-23e6-4758-9589-f09970f123ac None admin] RBAC: Authorizing `identity:get_role(role_id=9065c36fc305418baa5b977858b445dd)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:42.084312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1ac0-23e6-4758-9589-f09970f123ac None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:42.085894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 548/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:41 2022] GET /identity/v3/roles/9065c36fc305418baa5b977858b445dd => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:42.091500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcf562-d723-440c-84e2-9ed1b9ca3006 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:42.183515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcf562-d723-440c-84e2-9ed1b9ca3006 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:42.214374 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cf15d-a584-4cba-8de3-5a018947c071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:42.215238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cf15d-a584-4cba-8de3-5a018947c071 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:42.215519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cf15d-a584-4cba-8de3-5a018947c071 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:42.215793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cf15d-a584-4cba-8de3-5a018947c071 None admin] PATH_INFO: `/v3/users/74e030f719c347508ffd31db1bcaf101` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:42.229070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cf15d-a584-4cba-8de3-5a018947c071 None admin] RBAC: Authorizing `identity:get_user(user_id=74e030f719c347508ffd31db1bcaf101)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:42.230541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cf15d-a584-4cba-8de3-5a018947c071 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:42.232157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 566/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:42 2022] GET /identity/v3/users/74e030f719c347508ffd31db1bcaf101 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:42.238013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1ac0-23e6-4758-9589-f09970f123ac None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:42.331235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1ac0-23e6-4758-9589-f09970f123ac None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:42.363299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b831d-5cbb-494a-be5f-bbeee7547f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:42.363877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b831d-5cbb-494a-be5f-bbeee7547f73 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:42.364163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b831d-5cbb-494a-be5f-bbeee7547f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:42.364444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b831d-5cbb-494a-be5f-bbeee7547f73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:42.426613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b831d-5cbb-494a-be5f-bbeee7547f73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:42.428049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b831d-5cbb-494a-be5f-bbeee7547f73 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:42.519589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 549/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:42.525402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cf15d-a584-4cba-8de3-5a018947c071 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:42.618334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cf15d-a584-4cba-8de3-5a018947c071 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:42.649381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936177db-d81d-4f44-9181-b9b96e498bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:42.650254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936177db-d81d-4f44-9181-b9b96e498bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:42.650542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936177db-d81d-4f44-9181-b9b96e498bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:42.650821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936177db-d81d-4f44-9181-b9b96e498bb8 None admin] PATH_INFO: `/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:42.652504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936177db-d81d-4f44-9181-b9b96e498bb8 None admin] RBAC: Authorizing `identity:get_project(project_id=2453833aba8d45fb8c4c4ee1a28b4cec)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:42.653957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936177db-d81d-4f44-9181-b9b96e498bb8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:42.655609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 567/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:05:42 2022] GET /identity/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:42.661394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b831d-5cbb-494a-be5f-bbeee7547f73 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:42.755445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b831d-5cbb-494a-be5f-bbeee7547f73 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:42.789187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71757209-5361-4dfa-a175-df22dfeee847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:42.789867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757209-5361-4dfa-a175-df22dfeee847 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:42.790154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757209-5361-4dfa-a175-df22dfeee847 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:42.790437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71757209-5361-4dfa-a175-df22dfeee847 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:42.791265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71757209-5361-4dfa-a175-df22dfeee847 None admin] RBAC: Adding query filter params (role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=2453833aba8d45fb8c4c4ee1a28b4cec, user.id=74e030f719c347508ffd31db1bcaf101) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:42.791606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71757209-5361-4dfa-a175-df22dfeee847 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:42.793043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71757209-5361-4dfa-a175-df22dfeee847 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:42.798254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 550/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:42 2022] GET /identity/v3/role_assignments?user.id=74e030f719c347508ffd31db1bcaf101&scope.project.id=2453833aba8d45fb8c4c4ee1a28b4cec&role.id=9065c36fc305418baa5b977858b445dd => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:44.975281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837d883-09d2-4062-92b3-da32b1da294f None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:44.975822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837d883-09d2-4062-92b3-da32b1da294f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:44.975822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837d883-09d2-4062-92b3-da32b1da294f None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:44.977167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 568/1118] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:44.982470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9abe9-daff-41f1-8944-25cecf3b0213 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:44.982851 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9abe9-daff-41f1-8944-25cecf3b0213 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:44.983028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9abe9-daff-41f1-8944-25cecf3b0213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:44.999250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-11f9abe9-daff-41f1-8944-25cecf3b0213 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:45.000425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-11f9abe9-daff-41f1-8944-25cecf3b0213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:45.063071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 551/1119] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:45.085341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837d883-09d2-4062-92b3-da32b1da294f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:45.177841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837d883-09d2-4062-92b3-da32b1da294f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:45.209029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62803d-03be-4440-b72a-55fd7e594ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:45.209888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62803d-03be-4440-b72a-55fd7e594ebd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:45.210169 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62803d-03be-4440-b72a-55fd7e594ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:45.210442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62803d-03be-4440-b72a-55fd7e594ebd None admin] PATH_INFO: `/v3/roles/9065c36fc305418baa5b977858b445dd` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:45.211899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62803d-03be-4440-b72a-55fd7e594ebd None admin] RBAC: Authorizing `identity:get_role(role_id=9065c36fc305418baa5b977858b445dd)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:45.213277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62803d-03be-4440-b72a-55fd7e594ebd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:45.215155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 569/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:45 2022] GET /identity/v3/roles/9065c36fc305418baa5b977858b445dd => generated 223 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:45.220905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9abe9-daff-41f1-8944-25cecf3b0213 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:45.313059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9abe9-daff-41f1-8944-25cecf3b0213 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:45.344568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627a3c5-347d-4034-ad0d-23339d1cf86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:45.345417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627a3c5-347d-4034-ad0d-23339d1cf86b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:45.345697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627a3c5-347d-4034-ad0d-23339d1cf86b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:45.345996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627a3c5-347d-4034-ad0d-23339d1cf86b None admin] PATH_INFO: `/v3/users/74e030f719c347508ffd31db1bcaf101` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:45.347152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627a3c5-347d-4034-ad0d-23339d1cf86b None admin] RBAC: Authorizing `identity:get_user(user_id=74e030f719c347508ffd31db1bcaf101)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:45.348563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627a3c5-347d-4034-ad0d-23339d1cf86b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:45.350181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 552/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:45 2022] GET /identity/v3/users/74e030f719c347508ffd31db1bcaf101 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:45.355642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62803d-03be-4440-b72a-55fd7e594ebd None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:45.446853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62803d-03be-4440-b72a-55fd7e594ebd None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:45.477969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de77c5b-0fa8-4173-b44b-544c07b2496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:45.478810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77c5b-0fa8-4173-b44b-544c07b2496c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:45.479112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77c5b-0fa8-4173-b44b-544c07b2496c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:45.479379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77c5b-0fa8-4173-b44b-544c07b2496c None admin] PATH_INFO: `/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:45.480534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de77c5b-0fa8-4173-b44b-544c07b2496c None admin] RBAC: Authorizing `identity:get_project(project_id=2453833aba8d45fb8c4c4ee1a28b4cec)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:45.481980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de77c5b-0fa8-4173-b44b-544c07b2496c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:45.483648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 570/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:05:45 2022] GET /identity/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec => generated 311 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:45.489498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627a3c5-347d-4034-ad0d-23339d1cf86b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:45.583722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627a3c5-347d-4034-ad0d-23339d1cf86b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:45.614903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a96b59-7330-46c9-b529-9e15ab644ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:45.615774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a96b59-7330-46c9-b529-9e15ab644ba0 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:45.616056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a96b59-7330-46c9-b529-9e15ab644ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:45.616331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a96b59-7330-46c9-b529-9e15ab644ba0 None admin] PATH_INFO: `/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec/users/74e030f719c347508ffd31db1bcaf101/roles/9065c36fc305418baa5b977858b445dd` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:45.618538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a96b59-7330-46c9-b529-9e15ab644ba0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2453833aba8d45fb8c4c4ee1a28b4cec, user_id=74e030f719c347508ffd31db1bcaf101, role_id=9065c36fc305418baa5b977858b445dd)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:45.620510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a96b59-7330-46c9-b529-9e15ab644ba0 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:45.694439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 553/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:05:45 2022] PUT /identity/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec/users/74e030f719c347508ffd31db1bcaf101/roles/9065c36fc305418baa5b977858b445dd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:05:47.873675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3cd11-2d3b-4b06-b1e0-99f3ff192a7a None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:47.874217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3cd11-2d3b-4b06-b1e0-99f3ff192a7a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:47.874217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3cd11-2d3b-4b06-b1e0-99f3ff192a7a None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:47.875567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 571/1124] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:47.880675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929e10-d908-47e6-8a96-2f9f02534b78 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:47.881070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929e10-d908-47e6-8a96-2f9f02534b78 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:47.881240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929e10-d908-47e6-8a96-2f9f02534b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:47.898157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dd929e10-d908-47e6-8a96-2f9f02534b78 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:47.899337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dd929e10-d908-47e6-8a96-2f9f02534b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:47.962311 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 554/1125] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:47.985977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3cd11-2d3b-4b06-b1e0-99f3ff192a7a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:48.080428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3cd11-2d3b-4b06-b1e0-99f3ff192a7a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:48.111961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65879c86-7e18-4222-94d8-c960d464dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:48.112794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879c86-7e18-4222-94d8-c960d464dafc None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:48.113077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879c86-7e18-4222-94d8-c960d464dafc None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:48.113352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879c86-7e18-4222-94d8-c960d464dafc None admin] PATH_INFO: `/v3/roles/9065c36fc305418baa5b977858b445dd` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:48.114834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65879c86-7e18-4222-94d8-c960d464dafc None admin] RBAC: Authorizing `identity:get_role(role_id=9065c36fc305418baa5b977858b445dd)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:48.116197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65879c86-7e18-4222-94d8-c960d464dafc None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:48.117530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 572/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:47 2022] GET /identity/v3/roles/9065c36fc305418baa5b977858b445dd => generated 223 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:48.123829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929e10-d908-47e6-8a96-2f9f02534b78 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:48.215291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929e10-d908-47e6-8a96-2f9f02534b78 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:48.247928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6f492-8736-44a1-b84d-f32550367dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:48.248788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6f492-8736-44a1-b84d-f32550367dbe None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:48.249071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6f492-8736-44a1-b84d-f32550367dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:48.249347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6f492-8736-44a1-b84d-f32550367dbe None admin] PATH_INFO: `/v3/users/74e030f719c347508ffd31db1bcaf101` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:48.250547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6f492-8736-44a1-b84d-f32550367dbe None admin] RBAC: Authorizing `identity:get_user(user_id=74e030f719c347508ffd31db1bcaf101)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:48.251995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6f492-8736-44a1-b84d-f32550367dbe None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:48.253616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 555/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:48 2022] GET /identity/v3/users/74e030f719c347508ffd31db1bcaf101 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:48.259432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65879c86-7e18-4222-94d8-c960d464dafc None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:48.352468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65879c86-7e18-4222-94d8-c960d464dafc None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:48.384139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab44e80-712f-4739-a14e-3d2c365b1b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:48.384719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab44e80-712f-4739-a14e-3d2c365b1b47 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:48.385003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab44e80-712f-4739-a14e-3d2c365b1b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:48.385895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab44e80-712f-4739-a14e-3d2c365b1b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:48.448251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab44e80-712f-4739-a14e-3d2c365b1b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:48.449685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab44e80-712f-4739-a14e-3d2c365b1b47 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:48.541173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 573/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:48.546994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6f492-8736-44a1-b84d-f32550367dbe None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:48.640916 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6f492-8736-44a1-b84d-f32550367dbe None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:48.671907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c4ea8-e094-4165-a37c-8535b2305f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:48.672753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ea8-e094-4165-a37c-8535b2305f19 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:48.673039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ea8-e094-4165-a37c-8535b2305f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:48.673313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ea8-e094-4165-a37c-8535b2305f19 None admin] PATH_INFO: `/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:48.674459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c4ea8-e094-4165-a37c-8535b2305f19 None admin] RBAC: Authorizing `identity:get_project(project_id=2453833aba8d45fb8c4c4ee1a28b4cec)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:48.675908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c4ea8-e094-4165-a37c-8535b2305f19 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:48.677568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 556/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:05:48 2022] GET /identity/v3/projects/2453833aba8d45fb8c4c4ee1a28b4cec => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:48.684203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab44e80-712f-4739-a14e-3d2c365b1b47 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:48.777339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab44e80-712f-4739-a14e-3d2c365b1b47 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:48.810756 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b6162-c2ca-4540-8e0c-893fd8eec87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:48.811414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6162-c2ca-4540-8e0c-893fd8eec87b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:48.811700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6162-c2ca-4540-8e0c-893fd8eec87b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:48.812005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6162-c2ca-4540-8e0c-893fd8eec87b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:48.813064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b6162-c2ca-4540-8e0c-893fd8eec87b None admin] RBAC: Adding query filter params (role.id=9065c36fc305418baa5b977858b445dd, scope.project.id=2453833aba8d45fb8c4c4ee1a28b4cec, user.id=74e030f719c347508ffd31db1bcaf101) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:48.813381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b6162-c2ca-4540-8e0c-893fd8eec87b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:48.814835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b6162-c2ca-4540-8e0c-893fd8eec87b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:48.820410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 574/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:48 2022] GET /identity/v3/role_assignments?user.id=74e030f719c347508ffd31db1bcaf101&scope.project.id=2453833aba8d45fb8c4c4ee1a28b4cec&role.id=9065c36fc305418baa5b977858b445dd => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:51.034525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3994dd-e2dd-4be4-ab36-1f88aae21e06 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:51.035081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3994dd-e2dd-4be4-ab36-1f88aae21e06 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:51.035081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3994dd-e2dd-4be4-ab36-1f88aae21e06 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:51.036441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 557/1131] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:51.041607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9eada-c461-414c-a955-f7ae47e6e652 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:51.041998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9eada-c461-414c-a955-f7ae47e6e652 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:51.042191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9eada-c461-414c-a955-f7ae47e6e652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:51.058489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-7fb9eada-c461-414c-a955-f7ae47e6e652 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:51.059658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7fb9eada-c461-414c-a955-f7ae47e6e652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:51.121264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 575/1132] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:51.144240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3994dd-e2dd-4be4-ab36-1f88aae21e06 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:51.238938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3994dd-e2dd-4be4-ab36-1f88aae21e06 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:51.271112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda88db-79aa-4409-b081-bb9c622d3a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:51.271885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda88db-79aa-4409-b081-bb9c622d3a17 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:51.272169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda88db-79aa-4409-b081-bb9c622d3a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:51.272445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda88db-79aa-4409-b081-bb9c622d3a17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:51.273620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda88db-79aa-4409-b081-bb9c622d3a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:51.275069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda88db-79aa-4409-b081-bb9c622d3a17 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:51.277232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 558/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:51.282946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9eada-c461-414c-a955-f7ae47e6e652 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:51.374182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9eada-c461-414c-a955-f7ae47e6e652 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:51.405231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8b2c1-cd2b-4cb5-9dff-1603f764c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:51.405900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8b2c1-cd2b-4cb5-9dff-1603f764c760 None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:51.406182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8b2c1-cd2b-4cb5-9dff-1603f764c760 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:51.406476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8b2c1-cd2b-4cb5-9dff-1603f764c760 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:51.407155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8b2c1-cd2b-4cb5-9dff-1603f764c760 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:51.408567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8b2c1-cd2b-4cb5-9dff-1603f764c760 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:51.535318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 576/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:05:51 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:53.772874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87f702-90f3-4172-94fb-1723643e0165 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:53.773376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87f702-90f3-4172-94fb-1723643e0165 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:53.773376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87f702-90f3-4172-94fb-1723643e0165 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:53.774761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 559/1135] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:53.780182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607c8b8-1abf-461a-82e3-32427d6c7eb1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:53.780566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607c8b8-1abf-461a-82e3-32427d6c7eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:53.780734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607c8b8-1abf-461a-82e3-32427d6c7eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:53.796795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-9607c8b8-1abf-461a-82e3-32427d6c7eb1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:53.797991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9607c8b8-1abf-461a-82e3-32427d6c7eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:53.859459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 577/1136] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:05:53.883886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87f702-90f3-4172-94fb-1723643e0165 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:53.980559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87f702-90f3-4172-94fb-1723643e0165 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:54.012375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483607f5-d2c5-47c2-b4af-6bf13af2dd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:54.013164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483607f5-d2c5-47c2-b4af-6bf13af2dd2e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:54.013440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483607f5-d2c5-47c2-b4af-6bf13af2dd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:54.013716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483607f5-d2c5-47c2-b4af-6bf13af2dd2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:54.014914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483607f5-d2c5-47c2-b4af-6bf13af2dd2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:54.016349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483607f5-d2c5-47c2-b4af-6bf13af2dd2e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:54.017980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 560/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:05:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:54.023819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607c8b8-1abf-461a-82e3-32427d6c7eb1 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:54.115280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607c8b8-1abf-461a-82e3-32427d6c7eb1 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:54.146834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b417f-1068-457e-a056-463cf4658b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:54.147482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b417f-1068-457e-a056-463cf4658b7a None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:54.147764 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b417f-1068-457e-a056-463cf4658b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:54.148035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b417f-1068-457e-a056-463cf4658b7a None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:54.148734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b417f-1068-457e-a056-463cf4658b7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:54.150179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b417f-1068-457e-a056-463cf4658b7a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:54.242568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 578/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:05:54 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:05:56.479068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3595cc-d3d1-43c4-beff-a2da68ad087a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:56.479455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3595cc-d3d1-43c4-beff-a2da68ad087a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:56.479629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3595cc-d3d1-43c4-beff-a2da68ad087a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:56.481003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 561/1139] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:56.486128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48adf1-79df-4424-afdf-f31bcd4eb805 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:56.486505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48adf1-79df-4424-afdf-f31bcd4eb805 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:56.486683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48adf1-79df-4424-afdf-f31bcd4eb805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:56.503515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ae48adf1-79df-4424-afdf-f31bcd4eb805 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:56.504680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ae48adf1-79df-4424-afdf-f31bcd4eb805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:56.566157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 579/1140] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:56.589685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3595cc-d3d1-43c4-beff-a2da68ad087a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:56.684814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3595cc-d3d1-43c4-beff-a2da68ad087a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:56.716354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c51a6-b49b-4c15-a2b6-2b44363c6cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:56.717129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c51a6-b49b-4c15-a2b6-2b44363c6cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:56.717413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c51a6-b49b-4c15-a2b6-2b44363c6cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:56.717690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c51a6-b49b-4c15-a2b6-2b44363c6cf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:56.727351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c51a6-b49b-4c15-a2b6-2b44363c6cf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-110c51a6-b49b-4c15-a2b6-2b44363c6cf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:05:56.734948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:05:56.736010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:05:56.737046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 562/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:05:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:05:56.741927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48adf1-79df-4424-afdf-f31bcd4eb805 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:56.834794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48adf1-79df-4424-afdf-f31bcd4eb805 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:56.866002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:56.866660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:56.866947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:56.867226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:56.867966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:56.868272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:56.872073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:56.881776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 580/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:05:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:56.888833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c51a6-b49b-4c15-a2b6-2b44363c6cf8 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:56.985659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c51a6-b49b-4c15-a2b6-2b44363c6cf8 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:57.016731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71152f7-e622-4abb-8286-6a04c90c65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:57.017583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71152f7-e622-4abb-8286-6a04c90c65b6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:57.017894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71152f7-e622-4abb-8286-6a04c90c65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:57.018167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71152f7-e622-4abb-8286-6a04c90c65b6 None admin] PATH_INFO: `/v3/users/6609965872ba4acfb4bbb59b3ddae1b8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:57.030498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71152f7-e622-4abb-8286-6a04c90c65b6 None admin] RBAC: Authorizing `identity:get_user(user_id=6609965872ba4acfb4bbb59b3ddae1b8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:57.031918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71152f7-e622-4abb-8286-6a04c90c65b6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:57.033522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 563/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:05:56 2022] GET /identity/v3/users/6609965872ba4acfb4bbb59b3ddae1b8 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:57.039344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:57.133503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdea458-8170-4549-90be-fe2470c0c9c3 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:57.164262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32551f-6093-418c-a01a-f6431796bcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:57.164903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32551f-6093-418c-a01a-f6431796bcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:57.165190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32551f-6093-418c-a01a-f6431796bcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:57.165466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32551f-6093-418c-a01a-f6431796bcf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:57.227692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32551f-6093-418c-a01a-f6431796bcf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:57.229662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32551f-6093-418c-a01a-f6431796bcf4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:57.320436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 581/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:05:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:05:57.326355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71152f7-e622-4abb-8286-6a04c90c65b6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:57.428038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71152f7-e622-4abb-8286-6a04c90c65b6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:57.463534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b594d-5fd4-4f9a-84fb-d79a853bcf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:57.464416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b594d-5fd4-4f9a-84fb-d79a853bcf64 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:57.464693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b594d-5fd4-4f9a-84fb-d79a853bcf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:57.464965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b594d-5fd4-4f9a-84fb-d79a853bcf64 None admin] PATH_INFO: `/v3/projects/ff17a1c8d7b348f1835493b0ff90038b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:57.466637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b594d-5fd4-4f9a-84fb-d79a853bcf64 None admin] RBAC: Authorizing `identity:get_project(project_id=ff17a1c8d7b348f1835493b0ff90038b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:57.467572 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b594d-5fd4-4f9a-84fb-d79a853bcf64 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:57.469417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 564/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:05:57 2022] GET /identity/v3/projects/ff17a1c8d7b348f1835493b0ff90038b => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:57.475673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32551f-6093-418c-a01a-f6431796bcf4 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:05:57.579875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32551f-6093-418c-a01a-f6431796bcf4 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:05:57.616163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dec04c-9577-4790-bcc8-5109108736c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:05:57.616812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dec04c-9577-4790-bcc8-5109108736c3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:57.617097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dec04c-9577-4790-bcc8-5109108736c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:57.617374 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dec04c-9577-4790-bcc8-5109108736c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:57.618224 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dec04c-9577-4790-bcc8-5109108736c3 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=ff17a1c8d7b348f1835493b0ff90038b, user.id=6609965872ba4acfb4bbb59b3ddae1b8) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:05:57.618534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dec04c-9577-4790-bcc8-5109108736c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:05:57.619984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dec04c-9577-4790-bcc8-5109108736c3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:05:57.625286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 582/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:05:57 2022] GET /identity/v3/role_assignments?user.id=6609965872ba4acfb4bbb59b3ddae1b8&scope.project.id=ff17a1c8d7b348f1835493b0ff90038b&role.id=663341777b0b44128ed78fe8d40f6462 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:05:59.830255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae617cf-b4d6-4428-a0c0-3562e36c65ef None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:59.830794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae617cf-b4d6-4428-a0c0-3562e36c65ef None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:59.830794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae617cf-b4d6-4428-a0c0-3562e36c65ef None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:59.832296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 565/1147] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:05:59.838424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaa0a4-f483-40a9-84b6-e504e3c2fa3b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:05:59.838921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaa0a4-f483-40a9-84b6-e504e3c2fa3b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:05:59.839045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaa0a4-f483-40a9-84b6-e504e3c2fa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:05:59.856425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-fecaa0a4-f483-40a9-84b6-e504e3c2fa3b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:05:59.857637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-fecaa0a4-f483-40a9-84b6-e504e3c2fa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:05:59.923132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 583/1148] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:05:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:05:59.946049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae617cf-b4d6-4428-a0c0-3562e36c65ef None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:00.042618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae617cf-b4d6-4428-a0c0-3562e36c65ef None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:00.074559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c77c-ba8d-4c99-b10d-2143360d7c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:00.075336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c77c-ba8d-4c99-b10d-2143360d7c74 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:00.075615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c77c-ba8d-4c99-b10d-2143360d7c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:00.075880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c77c-ba8d-4c99-b10d-2143360d7c74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:00.085420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6c77c-ba8d-4c99-b10d-2143360d7c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-e4b6c77c-ba8d-4c99-b10d-2143360d7c74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:06:00.093472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:06:00.094600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:06:00.095772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 566/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:06:00.100586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecaa0a4-f483-40a9-84b6-e504e3c2fa3b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:00.194071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecaa0a4-f483-40a9-84b6-e504e3c2fa3b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:00.227434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:00.228098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:00.228402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:00.228675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:00.229414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:00.229720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:00.231107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:00.238763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 584/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:06:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:00.244819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c77c-ba8d-4c99-b10d-2143360d7c74 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:00.337019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c77c-ba8d-4c99-b10d-2143360d7c74 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:00.367828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775b37-013c-495c-a244-d1d615ae260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:00.368676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775b37-013c-495c-a244-d1d615ae260a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:00.369482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775b37-013c-495c-a244-d1d615ae260a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:00.369848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef775b37-013c-495c-a244-d1d615ae260a None admin] PATH_INFO: `/v3/users/6609965872ba4acfb4bbb59b3ddae1b8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:00.371014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef775b37-013c-495c-a244-d1d615ae260a None admin] RBAC: Authorizing `identity:get_user(user_id=6609965872ba4acfb4bbb59b3ddae1b8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:00.372425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef775b37-013c-495c-a244-d1d615ae260a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:00.374041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 567/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:06:00 2022] GET /identity/v3/users/6609965872ba4acfb4bbb59b3ddae1b8 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:00.379630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:00.473165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929fcc-7f43-4771-8ee6-8d312c1ff615 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:00.505208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683310a-b045-49b4-814f-11deb6622bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:00.506106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683310a-b045-49b4-814f-11deb6622bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:00.506391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683310a-b045-49b4-814f-11deb6622bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:00.506667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683310a-b045-49b4-814f-11deb6622bd4 None admin] PATH_INFO: `/v3/projects/ff17a1c8d7b348f1835493b0ff90038b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:00.507843 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683310a-b045-49b4-814f-11deb6622bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=ff17a1c8d7b348f1835493b0ff90038b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:00.509275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683310a-b045-49b4-814f-11deb6622bd4 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:00.511044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 585/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:06:00 2022] GET /identity/v3/projects/ff17a1c8d7b348f1835493b0ff90038b => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:00.517336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775b37-013c-495c-a244-d1d615ae260a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:00.611782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef775b37-013c-495c-a244-d1d615ae260a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:00.642343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626ac79-a193-47b3-b8ff-0f96d3a96b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:00.643215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ac79-a193-47b3-b8ff-0f96d3a96b4a None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:00.643499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ac79-a193-47b3-b8ff-0f96d3a96b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:00.643776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ac79-a193-47b3-b8ff-0f96d3a96b4a None admin] PATH_INFO: `/v3/projects/ff17a1c8d7b348f1835493b0ff90038b/users/6609965872ba4acfb4bbb59b3ddae1b8/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:00.645722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626ac79-a193-47b3-b8ff-0f96d3a96b4a None admin] RBAC: Authorizing `identity:create_grant(project_id=ff17a1c8d7b348f1835493b0ff90038b, user_id=6609965872ba4acfb4bbb59b3ddae1b8, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:00.647174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626ac79-a193-47b3-b8ff-0f96d3a96b4a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:00.722479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 568/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:06:00 2022] PUT /identity/v3/projects/ff17a1c8d7b348f1835493b0ff90038b/users/6609965872ba4acfb4bbb59b3ddae1b8/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:06:02.890740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c09c5-a408-44b5-ac04-9876342beaf8 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:02.891246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c09c5-a408-44b5-ac04-9876342beaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:02.891246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c09c5-a408-44b5-ac04-9876342beaf8 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:02.892578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 586/1154] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:06:02.897991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52045570-f0a9-4a2f-a138-4f9310aa1182 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:02.898404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52045570-f0a9-4a2f-a138-4f9310aa1182 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:02.898550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52045570-f0a9-4a2f-a138-4f9310aa1182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:02.914847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-52045570-f0a9-4a2f-a138-4f9310aa1182 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:06:02.916084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-52045570-f0a9-4a2f-a138-4f9310aa1182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:06:02.983312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 569/1155] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:06:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:06:03.006555 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c09c5-a408-44b5-ac04-9876342beaf8 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:03.101868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c09c5-a408-44b5-ac04-9876342beaf8 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:03.133431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73078e9e-7cd5-4b03-b5a3-d289c17233ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:03.134239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73078e9e-7cd5-4b03-b5a3-d289c17233ea None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:03.134551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73078e9e-7cd5-4b03-b5a3-d289c17233ea None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:03.134832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73078e9e-7cd5-4b03-b5a3-d289c17233ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:03.144826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73078e9e-7cd5-4b03-b5a3-d289c17233ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-73078e9e-7cd5-4b03-b5a3-d289c17233ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:06:03.152422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:06:03.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:06:03.154543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 587/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:06:03.159549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52045570-f0a9-4a2f-a138-4f9310aa1182 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:03.251155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52045570-f0a9-4a2f-a138-4f9310aa1182 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:03.281726 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:03.282400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:03.282679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:03.282951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:03.283687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:03.283987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:03.285343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:03.293368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 570/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:06:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:03.298936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73078e9e-7cd5-4b03-b5a3-d289c17233ea None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:03.391669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73078e9e-7cd5-4b03-b5a3-d289c17233ea None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:03.423659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6d127-6bc9-4d1f-8968-b14abf157082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:03.424467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d127-6bc9-4d1f-8968-b14abf157082 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:03.424748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d127-6bc9-4d1f-8968-b14abf157082 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:03.425025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d127-6bc9-4d1f-8968-b14abf157082 None admin] PATH_INFO: `/v3/users/6609965872ba4acfb4bbb59b3ddae1b8` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:03.426226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6d127-6bc9-4d1f-8968-b14abf157082 None admin] RBAC: Authorizing `identity:get_user(user_id=6609965872ba4acfb4bbb59b3ddae1b8)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:03.427650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6d127-6bc9-4d1f-8968-b14abf157082 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:03.429231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 588/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:06:03 2022] GET /identity/v3/users/6609965872ba4acfb4bbb59b3ddae1b8 => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:03.435216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:03.530269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c494bd7-358e-4721-aa67-db7ff57bbdea None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:03.561871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7337b2-0297-49ac-b8b7-44f06d3308ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:03.562507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7337b2-0297-49ac-b8b7-44f06d3308ad None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:03.562794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7337b2-0297-49ac-b8b7-44f06d3308ad None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:03.563073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7337b2-0297-49ac-b8b7-44f06d3308ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:03.624981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7337b2-0297-49ac-b8b7-44f06d3308ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:03.626461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7337b2-0297-49ac-b8b7-44f06d3308ad None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:03.716444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 571/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:06:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:06:03.722260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6d127-6bc9-4d1f-8968-b14abf157082 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:03.814599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6d127-6bc9-4d1f-8968-b14abf157082 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:03.847507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896583c1-5737-4024-bd43-e9012aada7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:03.848365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896583c1-5737-4024-bd43-e9012aada7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:03.848650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896583c1-5737-4024-bd43-e9012aada7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:03.848935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896583c1-5737-4024-bd43-e9012aada7a0 None admin] PATH_INFO: `/v3/projects/ff17a1c8d7b348f1835493b0ff90038b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:03.850242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896583c1-5737-4024-bd43-e9012aada7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=ff17a1c8d7b348f1835493b0ff90038b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:03.851680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896583c1-5737-4024-bd43-e9012aada7a0 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:03.853485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 589/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:06:03 2022] GET /identity/v3/projects/ff17a1c8d7b348f1835493b0ff90038b => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:03.859436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7337b2-0297-49ac-b8b7-44f06d3308ad None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:03.952586 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7337b2-0297-49ac-b8b7-44f06d3308ad None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:03.984285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef3a62-b93e-4a89-94bf-a826e337ae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:03.986653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef3a62-b93e-4a89-94bf-a826e337ae69 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:03.986653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef3a62-b93e-4a89-94bf-a826e337ae69 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:03.986653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef3a62-b93e-4a89-94bf-a826e337ae69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:03.986653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef3a62-b93e-4a89-94bf-a826e337ae69 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=ff17a1c8d7b348f1835493b0ff90038b, user.id=6609965872ba4acfb4bbb59b3ddae1b8) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:03.986653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef3a62-b93e-4a89-94bf-a826e337ae69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:03.987998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef3a62-b93e-4a89-94bf-a826e337ae69 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:03.993492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 572/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:06:03 2022] GET /identity/v3/role_assignments?user.id=6609965872ba4acfb4bbb59b3ddae1b8&scope.project.id=ff17a1c8d7b348f1835493b0ff90038b&role.id=663341777b0b44128ed78fe8d40f6462 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:06.216313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1caee-f509-48d2-b069-4c882264f570 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:06.216868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1caee-f509-48d2-b069-4c882264f570 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:06.216868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1caee-f509-48d2-b069-4c882264f570 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:06.218272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 590/1162] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:06:06.223412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919f97f-673f-4eeb-8551-04401754967b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:06.223786 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919f97f-673f-4eeb-8551-04401754967b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:06.223968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919f97f-673f-4eeb-8551-04401754967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:06.240092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8919f97f-673f-4eeb-8551-04401754967b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:06:06.241271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8919f97f-673f-4eeb-8551-04401754967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:06:06.304471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 573/1163] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:06:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:06:06.327835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1caee-f509-48d2-b069-4c882264f570 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:06.423454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1caee-f509-48d2-b069-4c882264f570 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:06.455760 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fee9c-6eac-49b4-b93f-e0e79857dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:06.456395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fee9c-6eac-49b4-b93f-e0e79857dac2 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:06.456678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fee9c-6eac-49b4-b93f-e0e79857dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:06.456954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fee9c-6eac-49b4-b93f-e0e79857dac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:06.523542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fee9c-6eac-49b4-b93f-e0e79857dac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:06.525015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fee9c-6eac-49b4-b93f-e0e79857dac2 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:06.614735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 591/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:06:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:06:06.621105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919f97f-673f-4eeb-8551-04401754967b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:06.713742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919f97f-673f-4eeb-8551-04401754967b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:06.745854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a83f7f-803b-4274-a496-0e039ea4e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:06.746604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a83f7f-803b-4274-a496-0e039ea4e140 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:06.746885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a83f7f-803b-4274-a496-0e039ea4e140 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:06.747160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a83f7f-803b-4274-a496-0e039ea4e140 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:06.753517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a83f7f-803b-4274-a496-0e039ea4e140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:06.754987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a83f7f-803b-4274-a496-0e039ea4e140 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-39a83f7f-803b-4274-a496-0e039ea4e140 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:06:06.768604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:06:06.769708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:06:06.770699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 574/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:06:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:06:06.775740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fee9c-6eac-49b4-b93f-e0e79857dac2 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:06.867676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fee9c-6eac-49b4-b93f-e0e79857dac2 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:06.899167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8aac4-9f73-4b4a-9aa0-1b9104516f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:06.899814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8aac4-9f73-4b4a-9aa0-1b9104516f03 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:06.900121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8aac4-9f73-4b4a-9aa0-1b9104516f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:06.900394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8aac4-9f73-4b4a-9aa0-1b9104516f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:06.901105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8aac4-9f73-4b4a-9aa0-1b9104516f03 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:06.901406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8aac4-9f73-4b4a-9aa0-1b9104516f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:06.902807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8aac4-9f73-4b4a-9aa0-1b9104516f03 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:06.909536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 592/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Fri May 13 13:06:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:09.098886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a2c2-55a3-439e-8227-06568c04275f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:09.099420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a2c2-55a3-439e-8227-06568c04275f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:09.099420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a2c2-55a3-439e-8227-06568c04275f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:09.100802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 575/1167] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:06:09.106461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bab3c-2c61-4c6e-a194-4db26fb037e0 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:09.106901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bab3c-2c61-4c6e-a194-4db26fb037e0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:09.107031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bab3c-2c61-4c6e-a194-4db26fb037e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:09.123808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6a4bab3c-2c61-4c6e-a194-4db26fb037e0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:06:09.124995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6a4bab3c-2c61-4c6e-a194-4db26fb037e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:06:09.187411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 593/1168] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:06:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:06:09.209525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074a2c2-55a3-439e-8227-06568c04275f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:09.304842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074a2c2-55a3-439e-8227-06568c04275f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:09.337981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0d2c2-8d3d-4d15-8b3f-ddc9f43726da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:09.338629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0d2c2-8d3d-4d15-8b3f-ddc9f43726da None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:09.338914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0d2c2-8d3d-4d15-8b3f-ddc9f43726da None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:09.339191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0d2c2-8d3d-4d15-8b3f-ddc9f43726da None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:09.339920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0d2c2-8d3d-4d15-8b3f-ddc9f43726da None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:09.341298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0d2c2-8d3d-4d15-8b3f-ddc9f43726da None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:09.454518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 576/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Fri May 13 13:06:09 2022] POST /identity/v3/domains => generated 267 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:06:11.688028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead48d4e-e950-445d-ad46-357855e4deae None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:11.688587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead48d4e-e950-445d-ad46-357855e4deae None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:11.688587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead48d4e-e950-445d-ad46-357855e4deae None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:11.690008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 594/1170] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:06:11.695282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5923675-db81-4568-984f-e9f44eadae7d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:11.695701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5923675-db81-4568-984f-e9f44eadae7d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:11.695850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5923675-db81-4568-984f-e9f44eadae7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:11.712596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f5923675-db81-4568-984f-e9f44eadae7d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:06:11.713814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f5923675-db81-4568-984f-e9f44eadae7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:06:11.776215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 577/1171] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:06:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:06:11.798741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead48d4e-e950-445d-ad46-357855e4deae None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:11.893621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead48d4e-e950-445d-ad46-357855e4deae None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:11.926099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209f30e-361c-475f-8f50-e434141e18ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:11.926958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209f30e-361c-475f-8f50-e434141e18ff None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:11.927239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209f30e-361c-475f-8f50-e434141e18ff None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:11.927514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209f30e-361c-475f-8f50-e434141e18ff None admin] PATH_INFO: `/v3/domains/61539c1f48f44f17a35941097962ef11` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:11.938321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209f30e-361c-475f-8f50-e434141e18ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=61539c1f48f44f17a35941097962ef11)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:11.939768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209f30e-361c-475f-8f50-e434141e18ff None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:11.941380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 595/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Fri May 13 13:06:11 2022] GET /identity/v3/domains/61539c1f48f44f17a35941097962ef11 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:11.947735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5923675-db81-4568-984f-e9f44eadae7d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:12.038045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5923675-db81-4568-984f-e9f44eadae7d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:12.070030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df54f4-2c18-4577-a8fb-1f7eb5f8dd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:12.070674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df54f4-2c18-4577-a8fb-1f7eb5f8dd03 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:12.070953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df54f4-2c18-4577-a8fb-1f7eb5f8dd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:12.071227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df54f4-2c18-4577-a8fb-1f7eb5f8dd03 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:12.072016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df54f4-2c18-4577-a8fb-1f7eb5f8dd03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:12.073439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df54f4-2c18-4577-a8fb-1f7eb5f8dd03 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:12.180149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 578/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:06:11 2022] POST /identity/v3/projects => generated 361 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:06:14.396924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac1031-f353-418c-b36f-37d1f94413b3 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:14.397482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac1031-f353-418c-b36f-37d1f94413b3 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:14.397482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac1031-f353-418c-b36f-37d1f94413b3 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:14.398846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 596/1174] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:06:14.404524 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45686-91dc-4074-8b54-b703188a3243 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:14.404909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45686-91dc-4074-8b54-b703188a3243 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:14.405086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45686-91dc-4074-8b54-b703188a3243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:14.421189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bbc45686-91dc-4074-8b54-b703188a3243 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:06:14.422387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bbc45686-91dc-4074-8b54-b703188a3243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:06:14.484955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 579/1175] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:06:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:06:14.508904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac1031-f353-418c-b36f-37d1f94413b3 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:14.605160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac1031-f353-418c-b36f-37d1f94413b3 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:14.636963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fbb99-5414-438d-9cd5-098512bdb7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:14.637823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fbb99-5414-438d-9cd5-098512bdb7aa None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:14.638173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fbb99-5414-438d-9cd5-098512bdb7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:14.638443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fbb99-5414-438d-9cd5-098512bdb7aa None admin] PATH_INFO: `/v3/domains/61539c1f48f44f17a35941097962ef11` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:14.639600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fbb99-5414-438d-9cd5-098512bdb7aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=61539c1f48f44f17a35941097962ef11)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:14.641025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fbb99-5414-438d-9cd5-098512bdb7aa None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:14.642650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 597/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Fri May 13 13:06:14 2022] GET /identity/v3/domains/61539c1f48f44f17a35941097962ef11 => generated 267 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:14.648388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45686-91dc-4074-8b54-b703188a3243 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:14.739941 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45686-91dc-4074-8b54-b703188a3243 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:14.771984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2898e91-3176-4f45-a80c-cd450d04839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:14.772861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2898e91-3176-4f45-a80c-cd450d04839c None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:14.773159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2898e91-3176-4f45-a80c-cd450d04839c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:14.773496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2898e91-3176-4f45-a80c-cd450d04839c None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:14.774344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2898e91-3176-4f45-a80c-cd450d04839c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:14.775951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2898e91-3176-4f45-a80c-cd450d04839c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:14.864730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 580/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:06:14 2022] POST /identity/v3/users => generated 311 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:06:17.090305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f844ea-b1bb-4d89-a63b-d706713d807b None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:17.090853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f844ea-b1bb-4d89-a63b-d706713d807b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:17.090853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f844ea-b1bb-4d89-a63b-d706713d807b None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:17.092212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 598/1178] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:06:17.097273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d146b-911a-4564-bc44-2e7113c498a4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:17.097648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d146b-911a-4564-bc44-2e7113c498a4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:17.097848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d146b-911a-4564-bc44-2e7113c498a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:17.114390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-329d146b-911a-4564-bc44-2e7113c498a4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:06:17.115561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-329d146b-911a-4564-bc44-2e7113c498a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:06:17.188145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 581/1179] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:06:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:06:17.211547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f844ea-b1bb-4d89-a63b-d706713d807b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:17.306982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f844ea-b1bb-4d89-a63b-d706713d807b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:17.339048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af745067-a9a8-4434-886d-66cca9a1a8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:17.339825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af745067-a9a8-4434-886d-66cca9a1a8ee None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:17.340111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af745067-a9a8-4434-886d-66cca9a1a8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:17.340434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af745067-a9a8-4434-886d-66cca9a1a8ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:17.350393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af745067-a9a8-4434-886d-66cca9a1a8ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-af745067-a9a8-4434-886d-66cca9a1a8ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:06:17.357885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:06:17.358975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:06:17.360188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 599/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:06:17.364983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d146b-911a-4564-bc44-2e7113c498a4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:17.456931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d146b-911a-4564-bc44-2e7113c498a4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:17.489400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:17.490105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:17.490385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:17.490656 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:17.491394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:17.491698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:17.493070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:17.500857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 582/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:17.506592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af745067-a9a8-4434-886d-66cca9a1a8ee None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:17.599800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af745067-a9a8-4434-886d-66cca9a1a8ee None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:17.631409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9413d35-f48a-44c7-ad2e-ac14734f4769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:17.632262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413d35-f48a-44c7-ad2e-ac14734f4769 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:17.632542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413d35-f48a-44c7-ad2e-ac14734f4769 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:17.632814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9413d35-f48a-44c7-ad2e-ac14734f4769 None admin] PATH_INFO: `/v3/users/833a6652c1da4b0f923317be210dffe1` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:17.645542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9413d35-f48a-44c7-ad2e-ac14734f4769 None admin] RBAC: Authorizing `identity:get_user(user_id=833a6652c1da4b0f923317be210dffe1)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:17.646999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9413d35-f48a-44c7-ad2e-ac14734f4769 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:17.648595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 600/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:06:17 2022] GET /identity/v3/users/833a6652c1da4b0f923317be210dffe1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:17.655249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:17.748474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dbd07-a04b-48ce-ba92-a495e7f2a1ad None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:17.779504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629c59-f68c-4c8b-87fc-15b3188a944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:17.780135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629c59-f68c-4c8b-87fc-15b3188a944e None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:17.780417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629c59-f68c-4c8b-87fc-15b3188a944e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:17.780690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629c59-f68c-4c8b-87fc-15b3188a944e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:17.843329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629c59-f68c-4c8b-87fc-15b3188a944e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:17.844771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629c59-f68c-4c8b-87fc-15b3188a944e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:17.936001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 583/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:06:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:06:17.941962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9413d35-f48a-44c7-ad2e-ac14734f4769 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:18.034942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9413d35-f48a-44c7-ad2e-ac14734f4769 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:18.067421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d23be3-6ec4-44fa-acbb-c856e7e7f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:18.068258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d23be3-6ec4-44fa-acbb-c856e7e7f338 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:18.068535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d23be3-6ec4-44fa-acbb-c856e7e7f338 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:18.068808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d23be3-6ec4-44fa-acbb-c856e7e7f338 None admin] PATH_INFO: `/v3/projects/cdd687f76e00425ba07b4a04b8508cc7` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:18.069985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d23be3-6ec4-44fa-acbb-c856e7e7f338 None admin] RBAC: Authorizing `identity:get_project(project_id=cdd687f76e00425ba07b4a04b8508cc7)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:18.071454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d23be3-6ec4-44fa-acbb-c856e7e7f338 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:18.073128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 601/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:06:17 2022] GET /identity/v3/projects/cdd687f76e00425ba07b4a04b8508cc7 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:18.079191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629c59-f68c-4c8b-87fc-15b3188a944e None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:18.170688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629c59-f68c-4c8b-87fc-15b3188a944e None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:18.202541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02452415-bb0d-4b40-b869-e67d69839a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:18.203126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02452415-bb0d-4b40-b869-e67d69839a59 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:18.203408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02452415-bb0d-4b40-b869-e67d69839a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:18.203682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02452415-bb0d-4b40-b869-e67d69839a59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:18.204497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02452415-bb0d-4b40-b869-e67d69839a59 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=cdd687f76e00425ba07b4a04b8508cc7, user.id=833a6652c1da4b0f923317be210dffe1) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:18.204808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02452415-bb0d-4b40-b869-e67d69839a59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:18.206259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02452415-bb0d-4b40-b869-e67d69839a59 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:18.211387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 584/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:06:18 2022] GET /identity/v3/role_assignments?user.id=833a6652c1da4b0f923317be210dffe1&scope.project.id=cdd687f76e00425ba07b4a04b8508cc7&role.id=663341777b0b44128ed78fe8d40f6462 => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:20.369161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf4045-23db-469a-bf59-fc235d30421d None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:20.369734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf4045-23db-469a-bf59-fc235d30421d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:20.369734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf4045-23db-469a-bf59-fc235d30421d None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:20.371092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 602/1186] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:06:20.376312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a2eb1-1daf-42ed-bd4d-167f13a6f0e3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:20.376711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a2eb1-1daf-42ed-bd4d-167f13a6f0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:20.376883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a2eb1-1daf-42ed-bd4d-167f13a6f0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:20.393506 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5c6a2eb1-1daf-42ed-bd4d-167f13a6f0e3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:06:20.394734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5c6a2eb1-1daf-42ed-bd4d-167f13a6f0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:06:20.457496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 585/1187] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:06:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:06:20.479968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf4045-23db-469a-bf59-fc235d30421d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:20.573350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf4045-23db-469a-bf59-fc235d30421d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:20.604338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de53a6f-4150-40ef-8823-f848b3f892f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:20.605109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53a6f-4150-40ef-8823-f848b3f892f7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:20.605391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53a6f-4150-40ef-8823-f848b3f892f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:20.605663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53a6f-4150-40ef-8823-f848b3f892f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:20.615041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de53a6f-4150-40ef-8823-f848b3f892f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-1de53a6f-4150-40ef-8823-f848b3f892f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:06:20.623035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:06:20.624128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:06:20.625256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 603/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:06:20.629995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a2eb1-1daf-42ed-bd4d-167f13a6f0e3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:20.724646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a2eb1-1daf-42ed-bd4d-167f13a6f0e3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:20.756157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:20.756812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:20.757095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:20.757371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:20.758134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:20.758439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:20.760450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:20.768027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 586/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:20.773837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de53a6f-4150-40ef-8823-f848b3f892f7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:20.866082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de53a6f-4150-40ef-8823-f848b3f892f7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:20.897477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797098-c1ba-4626-8649-fdc79a8f8a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:20.898298 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797098-c1ba-4626-8649-fdc79a8f8a9d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:20.898580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797098-c1ba-4626-8649-fdc79a8f8a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:20.898852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797098-c1ba-4626-8649-fdc79a8f8a9d None admin] PATH_INFO: `/v3/users/833a6652c1da4b0f923317be210dffe1` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:20.900008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797098-c1ba-4626-8649-fdc79a8f8a9d None admin] RBAC: Authorizing `identity:get_user(user_id=833a6652c1da4b0f923317be210dffe1)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:20.901433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797098-c1ba-4626-8649-fdc79a8f8a9d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:20.903037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 604/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:06:20 2022] GET /identity/v3/users/833a6652c1da4b0f923317be210dffe1 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:20.908525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:21.004084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46388f3-7e11-4829-a396-2137b5318646 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:21.035228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be1b16-7786-45e9-82f1-d6780a5c7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:21.036074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be1b16-7786-45e9-82f1-d6780a5c7a3f None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:21.036362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be1b16-7786-45e9-82f1-d6780a5c7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:21.036641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be1b16-7786-45e9-82f1-d6780a5c7a3f None admin] PATH_INFO: `/v3/projects/cdd687f76e00425ba07b4a04b8508cc7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:21.037829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be1b16-7786-45e9-82f1-d6780a5c7a3f None admin] RBAC: Authorizing `identity:get_project(project_id=cdd687f76e00425ba07b4a04b8508cc7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:21.039273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be1b16-7786-45e9-82f1-d6780a5c7a3f None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:21.041430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 587/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:06:20 2022] GET /identity/v3/projects/cdd687f76e00425ba07b4a04b8508cc7 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:21.047136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797098-c1ba-4626-8649-fdc79a8f8a9d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:21.142105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797098-c1ba-4626-8649-fdc79a8f8a9d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:21.177447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580a60b-780d-42c0-9fd0-267ac188c0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:21.178283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a60b-780d-42c0-9fd0-267ac188c0fa None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:21.178569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a60b-780d-42c0-9fd0-267ac188c0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:21.178848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a60b-780d-42c0-9fd0-267ac188c0fa None admin] PATH_INFO: `/v3/projects/cdd687f76e00425ba07b4a04b8508cc7/users/833a6652c1da4b0f923317be210dffe1/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:21.180830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580a60b-780d-42c0-9fd0-267ac188c0fa None admin] RBAC: Authorizing `identity:create_grant(project_id=cdd687f76e00425ba07b4a04b8508cc7, user_id=833a6652c1da4b0f923317be210dffe1, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:21.182301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580a60b-780d-42c0-9fd0-267ac188c0fa None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:21.265075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 605/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:06:21 2022] PUT /identity/v3/projects/cdd687f76e00425ba07b4a04b8508cc7/users/833a6652c1da4b0f923317be210dffe1/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:06:23.428208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4543e-555d-4a71-8eb9-67b741a3bd32 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:23.428825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4543e-555d-4a71-8eb9-67b741a3bd32 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:23.428825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4543e-555d-4a71-8eb9-67b741a3bd32 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:23.430176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 588/1193] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:06:23.435655 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f957-3408-402d-954d-c13f362d794a None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:23.436068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f957-3408-402d-954d-c13f362d794a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:23.436210 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17f957-3408-402d-954d-c13f362d794a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:23.452712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ca17f957-3408-402d-954d-c13f362d794a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:06:23.453903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ca17f957-3408-402d-954d-c13f362d794a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:06:23.516590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 606/1194] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:06:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:06:23.539696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4543e-555d-4a71-8eb9-67b741a3bd32 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:23.644470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4543e-555d-4a71-8eb9-67b741a3bd32 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:23.675774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b62baf-212b-4b34-847c-f3b514e33342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:23.676552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b62baf-212b-4b34-847c-f3b514e33342 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:23.676839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b62baf-212b-4b34-847c-f3b514e33342 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:23.677118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b62baf-212b-4b34-847c-f3b514e33342 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:23.687532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b62baf-212b-4b34-847c-f3b514e33342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-55b62baf-212b-4b34-847c-f3b514e33342 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:06:23.695096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:06:23.696151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:06:23.701740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 589/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:06:23.702816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17f957-3408-402d-954d-c13f362d794a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:23.794592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17f957-3408-402d-954d-c13f362d794a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:23.826757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:23.831717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:23.831717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:23.831717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:23.831717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:23.831717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:23.831717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:23.838123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 607/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:23.843824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b62baf-212b-4b34-847c-f3b514e33342 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:23.937966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b62baf-212b-4b34-847c-f3b514e33342 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:23.969207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6560-fc70-47dc-abc3-4dc40b85f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:23.970097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6560-fc70-47dc-abc3-4dc40b85f12b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:23.970381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6560-fc70-47dc-abc3-4dc40b85f12b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:23.970654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6560-fc70-47dc-abc3-4dc40b85f12b None admin] PATH_INFO: `/v3/users/833a6652c1da4b0f923317be210dffe1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:23.971832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f6560-fc70-47dc-abc3-4dc40b85f12b None admin] RBAC: Authorizing `identity:get_user(user_id=833a6652c1da4b0f923317be210dffe1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:23.973266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f6560-fc70-47dc-abc3-4dc40b85f12b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:23.974917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 590/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Fri May 13 13:06:23 2022] GET /identity/v3/users/833a6652c1da4b0f923317be210dffe1 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:23.981065 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:24.073317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe7b8e-d07e-43ee-a45d-b5a70eeb4e42 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:24.104596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a6180-575d-4513-b218-36cbef65521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:24.105234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a6180-575d-4513-b218-36cbef65521c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:24.105520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a6180-575d-4513-b218-36cbef65521c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:24.105821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a6180-575d-4513-b218-36cbef65521c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:24.167839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a6180-575d-4513-b218-36cbef65521c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:24.169289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a6180-575d-4513-b218-36cbef65521c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:24.261616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 608/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:06:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:06:24.267536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6560-fc70-47dc-abc3-4dc40b85f12b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:24.361226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6560-fc70-47dc-abc3-4dc40b85f12b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:24.391417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707dc31-ca02-4a14-9e15-0744e1943b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:24.392257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707dc31-ca02-4a14-9e15-0744e1943b1a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:24.392535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707dc31-ca02-4a14-9e15-0744e1943b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:24.392804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707dc31-ca02-4a14-9e15-0744e1943b1a None admin] PATH_INFO: `/v3/projects/cdd687f76e00425ba07b4a04b8508cc7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:24.393981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707dc31-ca02-4a14-9e15-0744e1943b1a None admin] RBAC: Authorizing `identity:get_project(project_id=cdd687f76e00425ba07b4a04b8508cc7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:24.396272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707dc31-ca02-4a14-9e15-0744e1943b1a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:24.397961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 591/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Fri May 13 13:06:24 2022] GET /identity/v3/projects/cdd687f76e00425ba07b4a04b8508cc7 => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:06:24.404203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a6180-575d-4513-b218-36cbef65521c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:06:24.497063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a6180-575d-4513-b218-36cbef65521c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:06:24.529230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d5883-f558-4236-92f4-121e57d517fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:06:24.529851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5883-f558-4236-92f4-121e57d517fd None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:06:24.530138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5883-f558-4236-92f4-121e57d517fd None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:06:24.530432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5883-f558-4236-92f4-121e57d517fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:06:24.531230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d5883-f558-4236-92f4-121e57d517fd None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=cdd687f76e00425ba07b4a04b8508cc7, user.id=833a6652c1da4b0f923317be210dffe1) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:06:24.531543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d5883-f558-4236-92f4-121e57d517fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:06:24.532963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d5883-f558-4236-92f4-121e57d517fd None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:06:24.538541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 609/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:06:24 2022] GET /identity/v3/role_assignments?user.id=833a6652c1da4b0f923317be210dffe1&scope.project.id=cdd687f76e00425ba07b4a04b8508cc7&role.id=663341777b0b44128ed78fe8d40f6462 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:07:52.965466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83273a-f7ca-4112-8fdb-35d0ac62875e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:52.966050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83273a-f7ca-4112-8fdb-35d0ac62875e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:52.966050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83273a-f7ca-4112-8fdb-35d0ac62875e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:52.967398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 592/1201] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:07:52.972484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792efb74-1c21-44b4-9dd1-6368208383b2 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:52.972911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792efb74-1c21-44b4-9dd1-6368208383b2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:52.973055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792efb74-1c21-44b4-9dd1-6368208383b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:52.990098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-792efb74-1c21-44b4-9dd1-6368208383b2 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:07:52.991883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-792efb74-1c21-44b4-9dd1-6368208383b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:07:53.054935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 610/1202] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:07:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:07:53.078991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83273a-f7ca-4112-8fdb-35d0ac62875e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:53.179952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83273a-f7ca-4112-8fdb-35d0ac62875e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:53.212152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:53.212932 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:53.213217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:53.213494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:53.223225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:53.224665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:07:53.243721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:07:53.244833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:07:53.246086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:07:53.246086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:07:53.246086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:07:53.246086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:07:53.246086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 593/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:07:53.250802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792efb74-1c21-44b4-9dd1-6368208383b2 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:53.341549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792efb74-1c21-44b4-9dd1-6368208383b2 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:53.372891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a287920-3638-40ea-a1bd-bc318f267874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:53.373544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a287920-3638-40ea-a1bd-bc318f267874 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:53.373845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a287920-3638-40ea-a1bd-bc318f267874 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:53.374119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a287920-3638-40ea-a1bd-bc318f267874 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:53.374828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a287920-3638-40ea-a1bd-bc318f267874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:07:53.375129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a287920-3638-40ea-a1bd-bc318f267874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:53.376485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a287920-3638-40ea-a1bd-bc318f267874 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:53.386903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 611/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:07:53.392465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:53.492296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24f4b8-1d06-4810-a628-913e73019a58 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:53.524480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9c769-2f37-477b-92d0-cf06d869e68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:53.525144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9c769-2f37-477b-92d0-cf06d869e68e None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:53.526848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9c769-2f37-477b-92d0-cf06d869e68e None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:53.526848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9c769-2f37-477b-92d0-cf06d869e68e None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:53.526848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9c769-2f37-477b-92d0-cf06d869e68e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:53.527887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9c769-2f37-477b-92d0-cf06d869e68e None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:53.605681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 594/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Fri May 13 13:07:53 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:07:55.814801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146f64-b5d1-4c23-971b-529363bb778c None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:55.815351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146f64-b5d1-4c23-971b-529363bb778c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:55.815351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e146f64-b5d1-4c23-971b-529363bb778c None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:55.816621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 612/1206] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:07:55.822013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9494730-eb7c-4d7b-a47b-2fbd96d597e7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:55.822445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9494730-eb7c-4d7b-a47b-2fbd96d597e7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:55.822578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9494730-eb7c-4d7b-a47b-2fbd96d597e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:55.839716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d9494730-eb7c-4d7b-a47b-2fbd96d597e7 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:07:55.840896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d9494730-eb7c-4d7b-a47b-2fbd96d597e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:07:55.904630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 595/1207] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:07:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:07:55.928276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e146f64-b5d1-4c23-971b-529363bb778c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:56.025337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e146f64-b5d1-4c23-971b-529363bb778c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:56.056120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f031-37f1-415a-b3ef-b49c382aedcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:56.056899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f031-37f1-415a-b3ef-b49c382aedcb None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:56.057178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f031-37f1-415a-b3ef-b49c382aedcb None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:56.057693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f031-37f1-415a-b3ef-b49c382aedcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:56.067603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175f031-37f1-415a-b3ef-b49c382aedcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-9175f031-37f1-415a-b3ef-b49c382aedcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:07:56.075571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:07:56.076724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:07:56.077889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 613/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:07:56.082643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9494730-eb7c-4d7b-a47b-2fbd96d597e7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:56.176522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9494730-eb7c-4d7b-a47b-2fbd96d597e7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:56.209114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:56.209828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:56.210116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:56.210393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:56.213887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:07:56.213887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:56.213887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:56.220326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 596/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:07:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:07:56.226650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f031-37f1-415a-b3ef-b49c382aedcb None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:56.332939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f031-37f1-415a-b3ef-b49c382aedcb None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:56.365425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:56.366244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:56.366528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:56.366821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:56.376524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:56.377991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:07:56.394894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:07:56.396310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:07:56.397945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:07:56.397945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:07:56.397945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:07:56.397945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:07:56.397945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 614/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:07:56.402434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:56.498257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ceaa3-6396-41ac-b856-012c9d62d5d1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:56.530476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:56.531135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:56.531421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:56.531702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:56.532494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:07:56.532800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:56.534200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:56.546141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 597/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:07:56.551744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:56.649352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252eaac-e921-4383-8ceb-de52bad4324c None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:56.682185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:56.683233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:56.683511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:56.683780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:56.690579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:56.692005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:07:56.706650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:07:56.707829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:07:56.708810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 615/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:07:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:07:56.714215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:56.808826 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230d2f-d938-4d2b-97ec-708434e4eecb None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:56.841556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:56.843171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:56.843171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:56.843171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:56.843598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:07:56.843905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:56.845329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:56.862555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 598/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Fri May 13 13:07:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:07:56.868194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:56.962914 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdb490-daaf-4095-b9c0-b432d59ca94d None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:56.994565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9523862-f05b-4119-89dd-cb92c82d1731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:56.995227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9523862-f05b-4119-89dd-cb92c82d1731 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:56.995506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9523862-f05b-4119-89dd-cb92c82d1731 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:56.995777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9523862-f05b-4119-89dd-cb92c82d1731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:57.058847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9523862-f05b-4119-89dd-cb92c82d1731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:57.060286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9523862-f05b-4119-89dd-cb92c82d1731 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:57.153095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 616/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:07:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:07:57.158891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:57.253865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e7ba0-a96c-48c7-8c0f-88839083d26a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:57.285513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:57.286319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:57.286603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:57.286880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:57.297246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:57.307379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:07:57.315955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:07:57.317068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:07:57.318382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:07:57.318382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:07:57.318382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:07:57.318382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:07:57.318382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 599/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:07:57.322747 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9523862-f05b-4119-89dd-cb92c82d1731 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:57.415882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9523862-f05b-4119-89dd-cb92c82d1731 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:57.447832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:57.448489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:57.448774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:57.449052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:57.449918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:07:57.450224 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:57.451593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:57.462381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 617/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:07:57.468111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:57.563122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2f68-0370-42ba-bc53-805068f7617b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:57.595203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:57.597442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:57.597442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:57.597442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:57.603815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:57.605258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:07:57.622495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:07:57.623582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:07:57.624698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 600/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:07:57.629650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:57.722456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4548e7-a5fe-4872-9703-3a0804fe8bec None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:57.753804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579110de-34bf-49ef-9383-b1199c1da2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:57.754461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579110de-34bf-49ef-9383-b1199c1da2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:57.754743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579110de-34bf-49ef-9383-b1199c1da2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:57.755013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579110de-34bf-49ef-9383-b1199c1da2c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:57.755782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579110de-34bf-49ef-9383-b1199c1da2c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:07:57.756108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579110de-34bf-49ef-9383-b1199c1da2c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:57.757578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579110de-34bf-49ef-9383-b1199c1da2c3 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:57.769940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 618/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:07:57.776597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:07:57.871855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba10e95-786b-4e82-8510-fbba63387f91 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:07:57.903053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f38194-3f83-4936-9ac7-3f1f8799d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:07:57.903688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38194-3f83-4936-9ac7-3f1f8799d2ef None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:07:57.903969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38194-3f83-4936-9ac7-3f1f8799d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:07:57.904245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38194-3f83-4936-9ac7-3f1f8799d2ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:07:57.905053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f38194-3f83-4936-9ac7-3f1f8799d2ef None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=310c5d3ba81044ac8f3f192728cc3e3b) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:07:57.905360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f38194-3f83-4936-9ac7-3f1f8799d2ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:07:57.906831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f38194-3f83-4936-9ac7-3f1f8799d2ef None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:07:57.911943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 601/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:07:57 2022] GET /identity/v3/role_assignments?user.id=310c5d3ba81044ac8f3f192728cc3e3b&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:00.121777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc70c5-f187-4309-b5e7-0fe607763cbf None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:00.122353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc70c5-f187-4309-b5e7-0fe607763cbf None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:00.122353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc70c5-f187-4309-b5e7-0fe607763cbf None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:00.123731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 619/1220] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:08:00.128789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d79a8-54ef-41cf-99e2-412358866e52 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:00.133958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d79a8-54ef-41cf-99e2-412358866e52 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:00.133958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d79a8-54ef-41cf-99e2-412358866e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:00.146284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-408d79a8-54ef-41cf-99e2-412358866e52 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:00.147458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-408d79a8-54ef-41cf-99e2-412358866e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:00.211355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 602/1221] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:08:00.233660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc70c5-f187-4309-b5e7-0fe607763cbf None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:00.331899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc70c5-f187-4309-b5e7-0fe607763cbf None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:00.364717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80fb91-235d-4880-8d7b-c6d2f3889897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:00.382438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80fb91-235d-4880-8d7b-c6d2f3889897 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:00.382438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80fb91-235d-4880-8d7b-c6d2f3889897 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:00.382438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80fb91-235d-4880-8d7b-c6d2f3889897 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:00.382438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80fb91-235d-4880-8d7b-c6d2f3889897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-ff80fb91-235d-4880-8d7b-c6d2f3889897 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:08:00.384594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:00.385525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:00.386407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 620/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:08:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:00.391916 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d79a8-54ef-41cf-99e2-412358866e52 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:00.485219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d79a8-54ef-41cf-99e2-412358866e52 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:00.517963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:00.518627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:00.518913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:00.519191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:00.519975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] RBAC: Adding query filter params (name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:00.520377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:00.521776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:00.529374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 603/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:08:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:00.535183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80fb91-235d-4880-8d7b-c6d2f3889897 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:00.629627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80fb91-235d-4880-8d7b-c6d2f3889897 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:00.663030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:00.663030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:00.663030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:00.663030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:00.672730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:00.674214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:00.691327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:00.692221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:00.693232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:00.693232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:00.693232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:00.693232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:00.693232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 621/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:00.698195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:00.794581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea365be4-2601-4be8-9afc-1c9c8b998460 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:00.826126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:00.826784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:00.827071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:00.827353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:00.828070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:00.828376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:00.829763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:00.841607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 604/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:00.847377 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:00.941898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a1929-221b-4b66-99de-a8312b722e1a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:00.973441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:00.974282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:00.974584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:00.975429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:00.982643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:00.984147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:08:00.998078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:00.999161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:01.000239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 622/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:01.004977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:01.105111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951f4e2-9686-408a-bd17-349989e8e52d None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:01.139127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:01.139794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:01.140078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:01.140405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:01.141326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:01.141635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:01.143092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:01.160257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 605/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Fri May 13 13:08:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:01.166722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:01.263722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfd1e4-8f21-45e2-8ead-c363b73bd050 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:01.296228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:01.297021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:01.297303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:01.297575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:01.308990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:01.311748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:01.328789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:01.329656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:01.331049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:01.331049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:01.331049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:01.331049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:01.331049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 623/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:01.337339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:01.437048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb358e5a-f100-4eca-b5bb-9fdd2b805359 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:01.470848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:01.471510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:01.471795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:01.472071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:01.472891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:01.473196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:01.474621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:01.486246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 606/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:01.492012 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:01.603574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600b612-f985-4b8c-af1f-4d2432e9d8da None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:01.641671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:01.642503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:01.642786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:01.643061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:01.649915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:01.651359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:08:01.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:01.666309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:01.667593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 624/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:01.673409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:01.774293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150b0b3-cfc8-4072-84eb-d62c447ac33c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:01.808052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725be97-5b13-4479-8e59-424c1082350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:01.808297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725be97-5b13-4479-8e59-424c1082350a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:01.808432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725be97-5b13-4479-8e59-424c1082350a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:01.808704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725be97-5b13-4479-8e59-424c1082350a None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:01.809484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725be97-5b13-4479-8e59-424c1082350a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:01.809806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725be97-5b13-4479-8e59-424c1082350a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:01.811319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725be97-5b13-4479-8e59-424c1082350a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:01.824304 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 607/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:08:01.830198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:01.936296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3d97e-a294-42df-9b85-9d487adb700a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:01.971011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9e50f-1e85-4c8a-993d-921844a03868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:01.971981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e50f-1e85-4c8a-993d-921844a03868 None admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:01.972260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e50f-1e85-4c8a-993d-921844a03868 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:01.972532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e50f-1e85-4c8a-993d-921844a03868 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/310c5d3ba81044ac8f3f192728cc3e3b/roles/3445c3dc3ab4448aaa19f3d511a884b8` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:01.987557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9e50f-1e85-4c8a-993d-921844a03868 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=310c5d3ba81044ac8f3f192728cc3e3b, role_id=3445c3dc3ab4448aaa19f3d511a884b8)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:01.989172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9e50f-1e85-4c8a-993d-921844a03868 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:02.108170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 625/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:08:01 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/310c5d3ba81044ac8f3f192728cc3e3b/roles/3445c3dc3ab4448aaa19f3d511a884b8 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:08:04.290734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e420e-2a06-4d26-8049-3fd96523cd7d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:04.291269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e420e-2a06-4d26-8049-3fd96523cd7d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:04.291269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e420e-2a06-4d26-8049-3fd96523cd7d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:04.292679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 608/1233] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:08:04.297969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02058-fdf1-495a-acac-c51eb68fab3f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:04.298389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02058-fdf1-495a-acac-c51eb68fab3f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:04.298539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02058-fdf1-495a-acac-c51eb68fab3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:04.316958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d9f02058-fdf1-495a-acac-c51eb68fab3f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:04.318212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d9f02058-fdf1-495a-acac-c51eb68fab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:04.390661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 626/1234] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:08:04.414522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e420e-2a06-4d26-8049-3fd96523cd7d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:04.510885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e420e-2a06-4d26-8049-3fd96523cd7d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:04.546462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fe7f5-5cae-4c96-9847-41dc114f424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:04.546462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fe7f5-5cae-4c96-9847-41dc114f424a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:04.546462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fe7f5-5cae-4c96-9847-41dc114f424a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:04.546462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fe7f5-5cae-4c96-9847-41dc114f424a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:04.555583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fe7f5-5cae-4c96-9847-41dc114f424a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-de1fe7f5-5cae-4c96-9847-41dc114f424a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:08:04.563554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:04.564622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:04.565951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 609/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Fri May 13 13:08:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:04.571114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f02058-fdf1-495a-acac-c51eb68fab3f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:04.664937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f02058-fdf1-495a-acac-c51eb68fab3f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:04.695894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:04.696562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:04.696848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:04.697126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:04.697904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] RBAC: Adding query filter params (name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:04.698216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:04.699591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:04.707664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 627/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Fri May 13 13:08:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:04.713869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fe7f5-5cae-4c96-9847-41dc114f424a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:04.808660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fe7f5-5cae-4c96-9847-41dc114f424a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:04.840856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:04.841635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:04.841948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:04.842226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:04.853037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:04.854516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:04.871359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:04.872411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:04.873660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:04.873660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:04.873660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:04.873660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:04.873660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 610/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:08:04.878340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:04.974044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdffb7-1d8f-453d-9cdd-c06b9551a88f None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:05.005438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:05.006640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:05.006928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:05.008527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:05.008527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:05.008819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:05.010300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:05.021089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 628/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:05.027798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:05.120686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7dce4-2b4a-4392-89b4-59adbfdd131a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:05.152307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:05.153089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:05.153368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:05.153640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:05.160750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:05.162232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:08:05.175794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:05.176923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:05.178148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 611/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:05.183211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:05.277221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11663ed-ef6c-4dea-9b2f-5b58fa96466b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:05.309478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:05.310181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:05.310463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:05.310735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:05.311569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:05.311874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:05.313306 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:05.330159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 629/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Fri May 13 13:08:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:05.336325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:05.431269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951dcec-1d8b-4f21-bbb0-48f5831542b6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:05.462828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d069ab-47e4-4a45-8ed9-835e51899d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:05.463464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d069ab-47e4-4a45-8ed9-835e51899d49 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:05.463750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d069ab-47e4-4a45-8ed9-835e51899d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:05.464030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d069ab-47e4-4a45-8ed9-835e51899d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:05.527969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d069ab-47e4-4a45-8ed9-835e51899d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:05.529423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d069ab-47e4-4a45-8ed9-835e51899d49 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:05.622662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 612/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:08:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:08:05.628304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:05.721889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e2052-84cc-46ba-a70b-371fbbabc75e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:05.753712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:05.754535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:05.754814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:05.755083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:05.765214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:05.766687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:05.783984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:05.790392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:05.791319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:05.791319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:05.791319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:05.791319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:05.791319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 630/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:05.791605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d069ab-47e4-4a45-8ed9-835e51899d49 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:05.884760 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d069ab-47e4-4a45-8ed9-835e51899d49 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:05.917742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:05.918426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:05.918703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:05.918975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:05.919685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:05.919985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:05.921348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:05.932763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 613/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:08:05.938353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:06.031480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6496ac-bafc-4b8a-98f9-1d4b1bcabe64 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:06.063890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:06.064668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:06.064951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:06.065225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:06.072155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:06.073596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:08:06.087309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:06.093915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:06.095033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 631/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:06.095619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:06.190195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c888-35e0-4dfa-a744-82bc7e90acd6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:06.222171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee487fb0-95d6-4949-afe2-52b1cd7e4d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:06.222824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee487fb0-95d6-4949-afe2-52b1cd7e4d41 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:06.223106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee487fb0-95d6-4949-afe2-52b1cd7e4d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:06.223381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee487fb0-95d6-4949-afe2-52b1cd7e4d41 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:06.224161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee487fb0-95d6-4949-afe2-52b1cd7e4d41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:06.224470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee487fb0-95d6-4949-afe2-52b1cd7e4d41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:06.225926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee487fb0-95d6-4949-afe2-52b1cd7e4d41 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:06.237237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 614/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:08:06.243755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:06.343604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d553900-36de-4a34-b27d-a5b05bedca7a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:06.376746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685c319-0643-44e6-9022-0d27369a3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:06.377401 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685c319-0643-44e6-9022-0d27369a3bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:06.377688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685c319-0643-44e6-9022-0d27369a3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:06.378004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685c319-0643-44e6-9022-0d27369a3bf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:06.378836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685c319-0643-44e6-9022-0d27369a3bf0 None admin] RBAC: Adding query filter params (role.id=3445c3dc3ab4448aaa19f3d511a884b8, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=310c5d3ba81044ac8f3f192728cc3e3b) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:06.379152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685c319-0643-44e6-9022-0d27369a3bf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:06.380579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685c319-0643-44e6-9022-0d27369a3bf0 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:06.386508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 632/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:08:06 2022] GET /identity/v3/role_assignments?user.id=310c5d3ba81044ac8f3f192728cc3e3b&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=3445c3dc3ab4448aaa19f3d511a884b8 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:08.636768 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90243dcd-88d2-4e27-a9cb-c07fbd007d2f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:08.637291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90243dcd-88d2-4e27-a9cb-c07fbd007d2f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:08.637384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90243dcd-88d2-4e27-a9cb-c07fbd007d2f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:08.638703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 615/1247] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:08:08.643860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecbf2b-4695-4a5f-83e8-3659047366b9 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:08.644267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecbf2b-4695-4a5f-83e8-3659047366b9 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:08.644459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecbf2b-4695-4a5f-83e8-3659047366b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:08.661335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-30ecbf2b-4695-4a5f-83e8-3659047366b9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:08.663122 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-30ecbf2b-4695-4a5f-83e8-3659047366b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:08.726406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 633/1248] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:08:08.749597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90243dcd-88d2-4e27-a9cb-c07fbd007d2f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:08.847482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90243dcd-88d2-4e27-a9cb-c07fbd007d2f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:08.879290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da214803-9112-4bf1-8332-67b4b1e692ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:08.880073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da214803-9112-4bf1-8332-67b4b1e692ff None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:08.880357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da214803-9112-4bf1-8332-67b4b1e692ff None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:08.881133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da214803-9112-4bf1-8332-67b4b1e692ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:08.890701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da214803-9112-4bf1-8332-67b4b1e692ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-da214803-9112-4bf1-8332-67b4b1e692ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:08:08.898927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:08.900023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:08.900861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 616/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:08:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:08.906135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecbf2b-4695-4a5f-83e8-3659047366b9 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:09.000813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecbf2b-4695-4a5f-83e8-3659047366b9 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:09.034190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:09.034849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:09.035128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:09.035400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:09.036131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:09.036432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:09.037819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:09.046117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 634/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:08:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:09.052179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da214803-9112-4bf1-8332-67b4b1e692ff None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:09.151637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da214803-9112-4bf1-8332-67b4b1e692ff None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:09.185205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:09.186016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:09.186355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:09.186677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:09.197852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:09.199297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:09.216600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:09.217814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:09.219114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:09.219114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:09.219114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:09.219114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:09.219114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 617/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:09.223761 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:09.317521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bc7d1-97e6-4df7-9652-8fef6aa3dd4e None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:09.349191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:09.350446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:09.350732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:09.351009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:09.351771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:09.352107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:09.353485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:09.364635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 635/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:09.370380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:09.464668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29586ca-6591-48b5-9a3b-fbf6987d4e9b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:09.496065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:09.496862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:09.497146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:09.497419 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:09.506197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:09.508121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:08:09.522373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:09.523515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:09.524542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 618/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:09.529204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:09.621108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30a811-3d71-4017-9395-d4072e9f0175 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:09.653574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:09.654274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:09.654556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:09.654831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:09.655656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:09.655963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:09.657378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:09.673450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 636/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Fri May 13 13:08:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:09.679158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:09.783533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b28b41-75f6-4afc-b03e-2aa3c00be6b6 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:09.820633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ebcfb-1d4f-4d7b-b06b-bd3c08f5045c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:09.821280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebcfb-1d4f-4d7b-b06b-bd3c08f5045c None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:09.821563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebcfb-1d4f-4d7b-b06b-bd3c08f5045c None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:09.821864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebcfb-1d4f-4d7b-b06b-bd3c08f5045c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:09.884285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ebcfb-1d4f-4d7b-b06b-bd3c08f5045c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:09.885663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ebcfb-1d4f-4d7b-b06b-bd3c08f5045c None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:09.978292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 619/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:08:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:08:09.983999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:10.077579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f766e8-eb36-4f87-be53-469574bc31a7 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:10.124683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:10.125240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:10.125522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:10.125816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:10.135730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:10.137089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:10.154113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:10.155214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:10.160733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:10.160733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:10.160733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:10.160733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:10.160733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 637/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:10.161090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ebcfb-1d4f-4d7b-b06b-bd3c08f5045c None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:10.254273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ebcfb-1d4f-4d7b-b06b-bd3c08f5045c None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:10.286714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:10.287366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:10.287651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:10.287929 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:10.288639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:10.288950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:10.290352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:10.301006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 620/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:10.306589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:10.405883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323a1db-ed07-41d7-a1b9-1e6c0970e017 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:10.442669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:10.443959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:10.444240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:10.444515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:10.451016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:10.452475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:08:10.466489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:10.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:10.468520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 638/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:08:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:10.473653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:10.568936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d8df-2a78-4250-90d2-355ffa585acf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:10.600960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbf1cf-d150-4ecc-8164-e1b2b8e5c930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:10.601615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf1cf-d150-4ecc-8164-e1b2b8e5c930 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:10.601933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf1cf-d150-4ecc-8164-e1b2b8e5c930 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:10.602214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbf1cf-d150-4ecc-8164-e1b2b8e5c930 None admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:10.602997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbf1cf-d150-4ecc-8164-e1b2b8e5c930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:10.603306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbf1cf-d150-4ecc-8164-e1b2b8e5c930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:10.604738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbf1cf-d150-4ecc-8164-e1b2b8e5c930 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:10.616786 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 621/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:08:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:10.622869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:10.717696 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63d9a4-5897-47ff-8208-2f524eceaccc None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:10.749798 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8ba4a-d91b-4fd8-9066-94df68357b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:10.750476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8ba4a-d91b-4fd8-9066-94df68357b7b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:10.750759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8ba4a-d91b-4fd8-9066-94df68357b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:10.751033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8ba4a-d91b-4fd8-9066-94df68357b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:10.751898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8ba4a-d91b-4fd8-9066-94df68357b7b None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=310c5d3ba81044ac8f3f192728cc3e3b) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:10.752206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8ba4a-d91b-4fd8-9066-94df68357b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:10.753629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8ba4a-d91b-4fd8-9066-94df68357b7b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:10.759411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 639/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:08:10 2022] GET /identity/v3/role_assignments?user.id=310c5d3ba81044ac8f3f192728cc3e3b&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=663341777b0b44128ed78fe8d40f6462 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:08:12.939929 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1db3-fc6c-4d73-8b06-8f9e894767a4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:12.940525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1db3-fc6c-4d73-8b06-8f9e894767a4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:12.940525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b1db3-fc6c-4d73-8b06-8f9e894767a4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:12.941766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 622/1261] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:08:12.947868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c704f-4410-4405-b22a-00a88b16d34f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:12.948308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c704f-4410-4405-b22a-00a88b16d34f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:12.948436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c704f-4410-4405-b22a-00a88b16d34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:12.964474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-059c704f-4410-4405-b22a-00a88b16d34f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:12.965647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-059c704f-4410-4405-b22a-00a88b16d34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:13.028002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 640/1262] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 13 13:08:13.050199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b1db3-fc6c-4d73-8b06-8f9e894767a4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:13.148743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b1db3-fc6c-4d73-8b06-8f9e894767a4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:13.181110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c93b37-7111-4a3d-9192-3f368150b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:13.181912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c93b37-7111-4a3d-9192-3f368150b6ce None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:13.182193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c93b37-7111-4a3d-9192-3f368150b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:13.182728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c93b37-7111-4a3d-9192-3f368150b6ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:13.192352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c93b37-7111-4a3d-9192-3f368150b6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-a2c93b37-7111-4a3d-9192-3f368150b6ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:08:13.200356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:13.201501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:13.202610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 623/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:08:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:13.207589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c704f-4410-4405-b22a-00a88b16d34f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:13.299826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c704f-4410-4405-b22a-00a88b16d34f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:13.331713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:13.332375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:13.332658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:13.332932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:13.333703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:13.334530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:13.335896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:13.343514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 641/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:08:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:13.349377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c93b37-7111-4a3d-9192-3f368150b6ce None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:13.457435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c93b37-7111-4a3d-9192-3f368150b6ce None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:13.489939 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:13.490729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:13.491010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:13.491282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:13.501361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:13.502873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:13.519782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:13.529814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:13.530962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:13.530962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:13.530962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:13.530962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:13.530962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 624/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:13.531322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:13.623088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f0887-ecf2-4c9a-8655-279360d5ff81 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:13.655156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:13.655870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:13.656159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:13.656438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:13.657153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:13.657510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:13.658916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:13.670689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 642/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:13.676260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:13.773162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ad80b-9569-4cbc-ba27-380417e69457 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:13.805399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:13.806344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:13.806629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:13.806908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:13.814433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:13.815880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:08:13.829670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:13.830790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:13.832061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 625/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:13.837298 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:13.930944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565d0e5-ed9b-44b8-93dd-7cbe5121395b None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:13.963259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:13.963916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:13.964195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:13.964468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:13.965281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:13.965590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:13.967028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:13.983601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 643/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Fri May 13 13:08:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:13.989446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:14.084813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28217d47-ccd7-4a98-82ff-280aa7b44713 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:14.115917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:14.116694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:14.116978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:14.117254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:14.127465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:14.128902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:14.145866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:14.147368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:14.148298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:14.148298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:14.148298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:14.148298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:14.148298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 626/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:14.154218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:14.247127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3d2d1-97fc-4e47-9f9f-7d359513bff1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:14.282941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:14.283593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:14.283925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:14.284207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:14.284931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:14.285277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:14.286670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:14.297283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 644/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:14.303569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:14.398354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d4cc5-5fda-4191-a13e-6fca1eaa55a1 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:14.429526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:14.430332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:14.430616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:14.430891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:14.437507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:14.440068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:08:14.456813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:14.457967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:14.460860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 627/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:14.464836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:14.560828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ca44-97f7-4636-b29e-142bf39fcb70 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:14.592350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bcbe9-ae68-45b6-9f27-82efd903249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:14.592997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bcbe9-ae68-45b6-9f27-82efd903249f None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:14.593277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bcbe9-ae68-45b6-9f27-82efd903249f None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:14.593551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bcbe9-ae68-45b6-9f27-82efd903249f None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:14.594354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bcbe9-ae68-45b6-9f27-82efd903249f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:14.594741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bcbe9-ae68-45b6-9f27-82efd903249f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:14.596079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bcbe9-ae68-45b6-9f27-82efd903249f None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:14.608167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 645/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:14.614069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:14.706610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe021d-7797-45da-8647-a98985c598e4 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:14.738538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22cdfc-3d46-4f91-9b63-0744a7ff0486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:14.739407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22cdfc-3d46-4f91-9b63-0744a7ff0486 None admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:14.739685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22cdfc-3d46-4f91-9b63-0744a7ff0486 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:14.739964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22cdfc-3d46-4f91-9b63-0744a7ff0486 None admin] PATH_INFO: `/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/310c5d3ba81044ac8f3f192728cc3e3b/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:14.741968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22cdfc-3d46-4f91-9b63-0744a7ff0486 None admin] RBAC: Authorizing `identity:create_grant(project_id=a4bf72e0f9914e149849e2fa09294797, user_id=310c5d3ba81044ac8f3f192728cc3e3b, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:14.743408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22cdfc-3d46-4f91-9b63-0744a7ff0486 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:14.851346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 628/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Fri May 13 13:08:14 2022] PUT /identity/v3/projects/a4bf72e0f9914e149849e2fa09294797/users/310c5d3ba81044ac8f3f192728cc3e3b/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:08:17.057297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85b8f-e701-45b7-8c3a-bacbd3d86cfd None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:17.058135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85b8f-e701-45b7-8c3a-bacbd3d86cfd None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:17.058236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85b8f-e701-45b7-8c3a-bacbd3d86cfd None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:17.059604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 646/1274] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:08:17.064679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbf4ae-5469-418f-8a7b-f9878ea96916 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:17.065123 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbf4ae-5469-418f-8a7b-f9878ea96916 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:17.065219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbf4ae-5469-418f-8a7b-f9878ea96916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:17.081945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-76bbf4ae-5469-418f-8a7b-f9878ea96916 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:17.083129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-76bbf4ae-5469-418f-8a7b-f9878ea96916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:17.146572 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 629/1275] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:08:17.169449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee85b8f-e701-45b7-8c3a-bacbd3d86cfd None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:17.265445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee85b8f-e701-45b7-8c3a-bacbd3d86cfd None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:17.297878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d3fab-7f8a-43ee-b9f4-4df69c766ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:17.301585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d3fab-7f8a-43ee-b9f4-4df69c766ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:17.301585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d3fab-7f8a-43ee-b9f4-4df69c766ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:17.301585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d3fab-7f8a-43ee-b9f4-4df69c766ad1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:17.309862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d3fab-7f8a-43ee-b9f4-4df69c766ad1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-4b4d3fab-7f8a-43ee-b9f4-4df69c766ad1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(role_id) May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 13 13:08:17.317326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:17.318910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:17.320133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 647/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Fri May 13 13:08:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 13 13:08:17.324959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbf4ae-5469-418f-8a7b-f9878ea96916 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:17.419121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbf4ae-5469-418f-8a7b-f9878ea96916 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:17.452539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:17.453208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:17.453539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:17.453848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:17.454601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:17.454910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:17.456279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:17.464001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 630/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Fri May 13 13:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:17.469746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d3fab-7f8a-43ee-b9f4-4df69c766ad1 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:17.564729 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d3fab-7f8a-43ee-b9f4-4df69c766ad1 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:17.596526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:17.597302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:17.597582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:17.597915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:17.608929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:17.610425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:17.626623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:17.627652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:17.628636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:17.628636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:17.628636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:17.628636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:17.628636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 648/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:17.633459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:17.727361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7700e-2d3d-4955-ad89-426a28c64e5b None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:17.758599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:17.759246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:17.759528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:17.759806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:17.760517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:17.760821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:17.762208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:17.773841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 631/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:17.779294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:17.878029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fc94e-c998-4764-bfd1-ffd52636d304 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:17.911779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:17.917032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:17.917032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:17.917032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:17.921471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:17.922961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_user(user_id) May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 13 13:08:17.937261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:17.938259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:17.939277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 649/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:17.944891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:18.039427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103ad4f-e452-41cb-8df4-70c51ffb236a None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:18.071590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:18.072246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:18.072522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:18.072798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:18.073613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:18.073944 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:18.075354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:18.092588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 632/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Fri May 13 13:08:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:18.098256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:18.192923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce287cb-c57a-4b9e-8d5d-e7f2254e4939 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:18.226989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790b7f-d58a-4493-b551-0cff6208859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:18.228370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790b7f-d58a-4493-b551-0cff6208859a None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:18.228370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790b7f-d58a-4493-b551-0cff6208859a None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:18.228370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790b7f-d58a-4493-b551-0cff6208859a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:18.291579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57790b7f-d58a-4493-b551-0cff6208859a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:18.293039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57790b7f-d58a-4493-b551-0cff6208859a None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:18.386154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 650/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Fri May 13 13:08:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 13 13:08:18.391969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:18.487118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94581c9-d6f4-4788-ad65-afcee523abf9 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:18.529447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:18.530269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:18.530549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:18.530818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:18.540243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:18.541666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 13 13:08:18.558298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:18.559409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:18.560575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 13 13:08:18.560575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 13 13:08:18.560575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 13 13:08:18.560575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:18.560575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 633/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:18.565125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790b7f-d58a-4493-b551-0cff6208859a None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:18.661194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790b7f-d58a-4493-b551-0cff6208859a None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:18.693648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:18.694327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:18.694614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:18.694889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:18.695607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:18.695914 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:18.697284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:18.708225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 651/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Fri May 13 13:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:18.714482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:18.807006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c811498-cb61-41d6-8ec9-9f19304d9fcf None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:18.839289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:18.840070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:18.841117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:18.841117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:18.846969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:18.848401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(project_id) May 13 13:08:18.861640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application with Lock( May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:18.862737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: ERROR keystone.server.flask.application May 13 13:08:18.863896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 634/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Fri May 13 13:08:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:18.869068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:18.961629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113324e-34e9-4a11-be32-fe02a844c049 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:18.992758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88fc65-dbad-487d-a12a-816e3adbb665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:18.993408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88fc65-dbad-487d-a12a-816e3adbb665 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:18.993692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88fc65-dbad-487d-a12a-816e3adbb665 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:18.994015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88fc65-dbad-487d-a12a-816e3adbb665 None admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:18.994795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88fc65-dbad-487d-a12a-816e3adbb665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:18.995105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88fc65-dbad-487d-a12a-816e3adbb665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:18.996526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88fc65-dbad-487d-a12a-816e3adbb665 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:19.008477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 652/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Fri May 13 13:08:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:19.014467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:19.109643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bf36a-0e26-40b4-85b1-c0f246635782 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:19.142126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa3925-d042-418d-a05f-763708bc7195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:19.142773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa3925-d042-418d-a05f-763708bc7195 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:19.143061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa3925-d042-418d-a05f-763708bc7195 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:19.143395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa3925-d042-418d-a05f-763708bc7195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:19.144200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa3925-d042-418d-a05f-763708bc7195 None admin] RBAC: Adding query filter params (role.id=663341777b0b44128ed78fe8d40f6462, scope.project.id=a4bf72e0f9914e149849e2fa09294797, user.id=310c5d3ba81044ac8f3f192728cc3e3b) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:19.144512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa3925-d042-418d-a05f-763708bc7195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:19.146507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa3925-d042-418d-a05f-763708bc7195 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:19.152118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 635/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Fri May 13 13:08:19 2022] GET /identity/v3/role_assignments?user.id=310c5d3ba81044ac8f3f192728cc3e3b&scope.project.id=a4bf72e0f9914e149849e2fa09294797&role.id=663341777b0b44128ed78fe8d40f6462 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:21.369689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea4ab8-db65-495d-b517-298a021176de None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:21.430517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea4ab8-db65-495d-b517-298a021176de None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:21.430517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea4ab8-db65-495d-b517-298a021176de None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:21.430517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 653/1288] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:08:21.430843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520edb0-7b8d-42d3-857b-dae6d92c4627 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:21.430843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520edb0-7b8d-42d3-857b-dae6d92c4627 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:21.430843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520edb0-7b8d-42d3-857b-dae6d92c4627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:21.430843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0520edb0-7b8d-42d3-857b-dae6d92c4627 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:21.430843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0520edb0-7b8d-42d3-857b-dae6d92c4627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:21.458132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 636/1289] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:08:21.480197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea4ab8-db65-495d-b517-298a021176de None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:21.578088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea4ab8-db65-495d-b517-298a021176de None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:21.610146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:21.610943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:21.611227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:21.611503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] PATH_INFO: `/v3/services/placement` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:21.612153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:21.613537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:21.625972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:21.627269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:21.628341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 654/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:08:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:21.635943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520edb0-7b8d-42d3-857b-dae6d92c4627 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:21.729094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520edb0-7b8d-42d3-857b-dae6d92c4627 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:21.761282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef79bd-4dc9-49c0-babf-d0c96818b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:21.761971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef79bd-4dc9-49c0-babf-d0c96818b294 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:21.762254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef79bd-4dc9-49c0-babf-d0c96818b294 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:21.762526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef79bd-4dc9-49c0-babf-d0c96818b294 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:21.763225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef79bd-4dc9-49c0-babf-d0c96818b294 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:21.763529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef79bd-4dc9-49c0-babf-d0c96818b294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:21.764906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef79bd-4dc9-49c0-babf-d0c96818b294 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:21.769765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 637/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Fri May 13 13:08:21 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:21.775890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:21.869025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d82f5-24eb-4c0c-b8b6-0151ddc67ece None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:21.900887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3d363-372e-461f-932f-bed7a35d8ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:21.901483 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d363-372e-461f-932f-bed7a35d8ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:21.901765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d363-372e-461f-932f-bed7a35d8ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:21.902075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d363-372e-461f-932f-bed7a35d8ca8 None admin] PATH_INFO: `/v3/services` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:21.902774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3d363-372e-461f-932f-bed7a35d8ca8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:21.903079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3d363-372e-461f-932f-bed7a35d8ca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:21.904455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3d363-372e-461f-932f-bed7a35d8ca8 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:21.908850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 655/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Fri May 13 13:08:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:24.080927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b6323-8e31-4587-b8e7-495c0668494d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:24.081480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b6323-8e31-4587-b8e7-495c0668494d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:24.081480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b6323-8e31-4587-b8e7-495c0668494d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:24.082871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 638/1293] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:08:24.088592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a67ba8-572d-493e-ac13-ae7594c2e36f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:24.089032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a67ba8-572d-493e-ac13-ae7594c2e36f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:24.089158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a67ba8-572d-493e-ac13-ae7594c2e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:24.106041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-30a67ba8-572d-493e-ac13-ae7594c2e36f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:24.107227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-30a67ba8-572d-493e-ac13-ae7594c2e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:24.172288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 656/1294] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:08:24.194979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b6323-8e31-4587-b8e7-495c0668494d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:24.295576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b6323-8e31-4587-b8e7-495c0668494d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:24.327102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d20f5-4c7c-4668-85c0-ae8bd5c67542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:24.327762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d20f5-4c7c-4668-85c0-ae8bd5c67542 None admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:24.328045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d20f5-4c7c-4668-85c0-ae8bd5c67542 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:24.328337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d20f5-4c7c-4668-85c0-ae8bd5c67542 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:24.329047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d20f5-4c7c-4668-85c0-ae8bd5c67542 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:24.330433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d20f5-4c7c-4668-85c0-ae8bd5c67542 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:24.719720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 639/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Fri May 13 13:08:24 2022] POST /identity/v3/services => generated 246 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:08:26.940265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da5ae6-11f6-4f02-a10d-96702e388c77 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:26.940810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da5ae6-11f6-4f02-a10d-96702e388c77 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:26.940810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da5ae6-11f6-4f02-a10d-96702e388c77 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:26.942173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 657/1296] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:08:26.948042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a601c52-a358-4666-bffc-05418271ba5c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:26.948455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a601c52-a358-4666-bffc-05418271ba5c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:26.948604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a601c52-a358-4666-bffc-05418271ba5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:26.965353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6a601c52-a358-4666-bffc-05418271ba5c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:26.966576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6a601c52-a358-4666-bffc-05418271ba5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:27.034310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 640/1297] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:08:27.056600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da5ae6-11f6-4f02-a10d-96702e388c77 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:27.151996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da5ae6-11f6-4f02-a10d-96702e388c77 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:27.184468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:27.185248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:27.185530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:27.185822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:27.186472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:27.187841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:27.199259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:27.200475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:27.201341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 658/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:08:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:27.206588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a601c52-a358-4666-bffc-05418271ba5c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:27.305753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a601c52-a358-4666-bffc-05418271ba5c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:27.338341 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:27.339000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:27.339284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:27.339603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:27.340310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:27.340618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:27.342004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:27.347019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 641/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Fri May 13 13:08:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:27.353567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:27.448297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3b3bb-40f1-4bd3-933c-d03ba202abd5 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:27.480009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84260d47-4a6d-42bd-84f7-5667557b14f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:27.480666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260d47-4a6d-42bd-84f7-5667557b14f0 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:27.480955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260d47-4a6d-42bd-84f7-5667557b14f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:27.481239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260d47-4a6d-42bd-84f7-5667557b14f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:27.482061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84260d47-4a6d-42bd-84f7-5667557b14f0 None admin] RBAC: Adding query filter params (interface=public, service_id=5c19c48228ea4fcfa13971dce4c184b3, region_id=RegionOne) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:27.482377 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84260d47-4a6d-42bd-84f7-5667557b14f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:27.483796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84260d47-4a6d-42bd-84f7-5667557b14f0 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:27.488755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 659/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Fri May 13 13:08:27 2022] GET /identity/v3/endpoints?service_id=5c19c48228ea4fcfa13971dce4c184b3&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:27.494330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:27.588123 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10f039-6b49-421d-8324-66384cab7ed5 None admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:27.625072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c61f1-7686-4887-a859-ec6e71edc91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:27.625733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c61f1-7686-4887-a859-ec6e71edc91d None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:27.626050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c61f1-7686-4887-a859-ec6e71edc91d None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:27.626323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c61f1-7686-4887-a859-ec6e71edc91d None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:27.626998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c61f1-7686-4887-a859-ec6e71edc91d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:27.628372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c61f1-7686-4887-a859-ec6e71edc91d None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:27.635766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 642/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Fri May 13 13:08:27 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:08:29.844153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414e3b9-d89c-4d42-9136-6c95af0ead6e None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:29.844656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414e3b9-d89c-4d42-9136-6c95af0ead6e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:29.844656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414e3b9-d89c-4d42-9136-6c95af0ead6e None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:29.846171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 660/1302] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:08:29.851921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4283-f0d2-4416-b8a6-d1a9c0c61059 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:29.852322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4283-f0d2-4416-b8a6-d1a9c0c61059 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:29.852473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c4283-f0d2-4416-b8a6-d1a9c0c61059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:29.869285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b25c4283-f0d2-4416-b8a6-d1a9c0c61059 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:08:29.877653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b25c4283-f0d2-4416-b8a6-d1a9c0c61059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:08:29.933316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 643/1303] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:08:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 13 13:08:29.955375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414e3b9-d89c-4d42-9136-6c95af0ead6e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:30.053699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414e3b9-d89c-4d42-9136-6c95af0ead6e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:30.086968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:30.087757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:30.088043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:30.088319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:30.088966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:30.090371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application Traceback (most recent call last): May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 13 13:08:30.102918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._get_service(service_id) May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 13 13:08:30.108922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.get_or_create( May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application with Lock( May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self._enter() May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application return self.creator() May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application created_value = creator( May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: ERROR keystone.server.flask.application May 13 13:08:30.110072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 661/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Fri May 13 13:08:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 13 13:08:30.111074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c4283-f0d2-4416-b8a6-d1a9c0c61059 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:30.200404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c4283-f0d2-4416-b8a6-d1a9c0c61059 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:30.232278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ec74-a4ac-4bbf-a322-2e85082c33bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:30.232949 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ec74-a4ac-4bbf-a322-2e85082c33bc None admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:30.233227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ec74-a4ac-4bbf-a322-2e85082c33bc None admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:30.233499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ec74-a4ac-4bbf-a322-2e85082c33bc None admin] PATH_INFO: `/v3/services` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:30.234215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ec74-a4ac-4bbf-a322-2e85082c33bc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:08:30.234516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ec74-a4ac-4bbf-a322-2e85082c33bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:30.235878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ec74-a4ac-4bbf-a322-2e85082c33bc None admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:30.240749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 644/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Fri May 13 13:08:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:08:30.246464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:08:30.346796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dadfe-6529-4f61-b344-ef9d7a1a3f37 None admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:08:30.378553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205f22d-ffb5-4c71-99d8-17e5845dc26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:08:30.379116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205f22d-ffb5-4c71-99d8-17e5845dc26e None admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:08:30.379397 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205f22d-ffb5-4c71-99d8-17e5845dc26e None admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:08:30.379669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205f22d-ffb5-4c71-99d8-17e5845dc26e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:08:30.380394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205f22d-ffb5-4c71-99d8-17e5845dc26e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:08:30.381750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205f22d-ffb5-4c71-99d8-17e5845dc26e None admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:08:30.508369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 662/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Fri May 13 13:08:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:16:17.527159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeaa38-e6b0-4ad4-b71b-db9658b5a3a2 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:17.528226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeaa38-e6b0-4ad4-b71b-db9658b5a3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:17.528721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeaa38-e6b0-4ad4-b71b-db9658b5a3a2 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:17.535886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ea401-7221-4e93-bf72-5427f53cd673 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:17.543486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ea401-7221-4e93-bf72-5427f53cd673 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:17.543486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ea401-7221-4e93-bf72-5427f53cd673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:17.543704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 645/1307] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:17 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:17.610622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-172ea401-7221-4e93-bf72-5427f53cd673 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:17.612204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-172ea401-7221-4e93-bf72-5427f53cd673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:17.738150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 663/1308] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:16:17.780734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76602a-d3ad-4120-b5e0-87f38d473585 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:17.781360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76602a-d3ad-4120-b5e0-87f38d473585 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:17.781873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76602a-d3ad-4120-b5e0-87f38d473585 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:17.783765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 646/1309] 10.222.0.25 () {54 vars in 964 bytes} [Fri May 13 13:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:17.790341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79432-6b68-486a-b04b-3bd5bfbcc139 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:17.791030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79432-6b68-486a-b04b-3bd5bfbcc139 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:17.791711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a79432-6b68-486a-b04b-3bd5bfbcc139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:17.975350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-74a79432-6b68-486a-b04b-3bd5bfbcc139 None None] MFA Rules not processed for user `8ddb0dcaa5304016af1bc1f88845fb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:17.979765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-74a79432-6b68-486a-b04b-3bd5bfbcc139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:18.096171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 664/1310] 10.222.0.25 () {58 vars in 1061 bytes} [Fri May 13 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:18.105919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76602a-d3ad-4120-b5e0-87f38d473585 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:18.107662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ab76602a-d3ad-4120-b5e0-87f38d473585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:18.228003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76602a-d3ad-4120-b5e0-87f38d473585 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:18.300294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d5b93-c2ee-4959-8d94-0d44bab4f195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:18.300946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5b93-c2ee-4959-8d94-0d44bab4f195 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:18.301233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5b93-c2ee-4959-8d94-0d44bab4f195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:18.301508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5b93-c2ee-4959-8d94-0d44bab4f195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:18.303231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-207d5b93-c2ee-4959-8d94-0d44bab4f195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:18.412526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d5b93-c2ee-4959-8d94-0d44bab4f195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:18.414039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d5b93-c2ee-4959-8d94-0d44bab4f195 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:18.434064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 647/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:21.049077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778244c-4758-4124-b22a-900b162c125c None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:21.049077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778244c-4758-4124-b22a-900b162c125c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:21.049077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778244c-4758-4124-b22a-900b162c125c None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:21.049077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 665/1312] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:21.051566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aaa9-114b-42bd-892b-a30dea203e3d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:21.051566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aaa9-114b-42bd-892b-a30dea203e3d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:21.051566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1aaa9-114b-42bd-892b-a30dea203e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:21.051566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a9a1aaa9-114b-42bd-892b-a30dea203e3d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:21.051566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a9a1aaa9-114b-42bd-892b-a30dea203e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:21.165538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 648/1313] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:21.203507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778244c-4758-4124-b22a-900b162c125c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:21.205132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3778244c-4758-4124-b22a-900b162c125c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:21.306769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778244c-4758-4124-b22a-900b162c125c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:21.376727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a9e6-0876-4ad9-a527-4ba1a70570fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:21.377575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a9e6-0876-4ad9-a527-4ba1a70570fd service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:21.377990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a9e6-0876-4ad9-a527-4ba1a70570fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:21.378367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a9e6-0876-4ad9-a527-4ba1a70570fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:21.380087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-35c8a9e6-0876-4ad9-a527-4ba1a70570fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:21.486857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8a9e6-0876-4ad9-a527-4ba1a70570fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:21.488305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8a9e6-0876-4ad9-a527-4ba1a70570fd service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:21.506602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 666/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:24.264752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850735a-91e8-4bfc-b912-2b1b7f7f03ff None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:24.265227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850735a-91e8-4bfc-b912-2b1b7f7f03ff None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:24.265327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850735a-91e8-4bfc-b912-2b1b7f7f03ff None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:24.266717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 649/1315] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:24.272098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f6586-a26a-44a9-897e-a3d364ec63c2 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:24.272555 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f6586-a26a-44a9-897e-a3d364ec63c2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:24.273309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f6586-a26a-44a9-897e-a3d364ec63c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:24.336986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-5e4f6586-a26a-44a9-897e-a3d364ec63c2 None None] MFA Rules not processed for user `1ff1944c64d841c3a8596ed53714d48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:24.338211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5e4f6586-a26a-44a9-897e-a3d364ec63c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:24.431859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 667/1316] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:24.485382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d0fd5-d04a-4fd5-a8f1-a5a3b54732c8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:24.485676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d0fd5-d04a-4fd5-a8f1-a5a3b54732c8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:24.485999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d0fd5-d04a-4fd5-a8f1-a5a3b54732c8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:24.487374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 650/1317] 10.222.0.25 () {54 vars in 964 bytes} [Fri May 13 13:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:24.494397 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41284-afe2-4a91-89a7-fcc4a26ab872 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:24.494715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41284-afe2-4a91-89a7-fcc4a26ab872 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:24.494985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc41284-afe2-4a91-89a7-fcc4a26ab872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:24.566584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-edc41284-afe2-4a91-89a7-fcc4a26ab872 None None] MFA Rules not processed for user `8ddb0dcaa5304016af1bc1f88845fb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:24.567686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-edc41284-afe2-4a91-89a7-fcc4a26ab872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:24.638113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 668/1318] 10.222.0.25 () {58 vars in 1061 bytes} [Fri May 13 13:16:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:16:24.647234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d0fd5-d04a-4fd5-a8f1-a5a3b54732c8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:24.648901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7a2d0fd5-d04a-4fd5-a8f1-a5a3b54732c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:24.748142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d0fd5-d04a-4fd5-a8f1-a5a3b54732c8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:24.818688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef08fc-e46f-4e3c-abb2-00a1fb921420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:24.819273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef08fc-e46f-4e3c-abb2-00a1fb921420 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:24.820142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef08fc-e46f-4e3c-abb2-00a1fb921420 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:24.820428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef08fc-e46f-4e3c-abb2-00a1fb921420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:24.822102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-04ef08fc-e46f-4e3c-abb2-00a1fb921420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:24.925742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef08fc-e46f-4e3c-abb2-00a1fb921420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:24.927231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef08fc-e46f-4e3c-abb2-00a1fb921420 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:24.945206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 651/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:28.395700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491285-2417-4d14-9ae1-5089ec367b02 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:28.396275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491285-2417-4d14-9ae1-5089ec367b02 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:28.396275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491285-2417-4d14-9ae1-5089ec367b02 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:28.397593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 669/1320] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:28.402927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1f2c4-5517-4bef-8aca-756628c93d9a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:28.403374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1f2c4-5517-4bef-8aca-756628c93d9a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:28.403471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1f2c4-5517-4bef-8aca-756628c93d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:28.465919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f7b1f2c4-5517-4bef-8aca-756628c93d9a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:28.467130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f7b1f2c4-5517-4bef-8aca-756628c93d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:28.559233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 652/1321] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:16:28.597971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5491285-2417-4d14-9ae1-5089ec367b02 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:28.599479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b5491285-2417-4d14-9ae1-5089ec367b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:28.698353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5491285-2417-4d14-9ae1-5089ec367b02 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:28.766953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331e8e1-4756-4381-b553-425bd71fef76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:28.767598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e8e1-4756-4381-b553-425bd71fef76 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:28.767884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e8e1-4756-4381-b553-425bd71fef76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:28.768161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331e8e1-4756-4381-b553-425bd71fef76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:28.769811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f331e8e1-4756-4381-b553-425bd71fef76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:28.874844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331e8e1-4756-4381-b553-425bd71fef76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:28.877701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331e8e1-4756-4381-b553-425bd71fef76 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:28.896749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 670/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:32.068716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d549-edcb-41a3-860e-f81d999c9bc8 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:32.069278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d549-edcb-41a3-860e-f81d999c9bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:32.070005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d549-edcb-41a3-860e-f81d999c9bc8 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:32.071247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 653/1323] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:32.076524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d1fab-b2af-4640-9dbb-8bf718da395e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:32.076946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d1fab-b2af-4640-9dbb-8bf718da395e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:32.077069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d1fab-b2af-4640-9dbb-8bf718da395e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:32.140284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2c8d1fab-b2af-4640-9dbb-8bf718da395e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:32.141494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2c8d1fab-b2af-4640-9dbb-8bf718da395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:32.237001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 671/1324] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:32.273132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d549-edcb-41a3-860e-f81d999c9bc8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:32.274917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-10b3d549-edcb-41a3-860e-f81d999c9bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:32.375439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d549-edcb-41a3-860e-f81d999c9bc8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:32.445778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caad929-6ccc-47ce-b28f-1085abbbd0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:32.446455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caad929-6ccc-47ce-b28f-1085abbbd0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:32.446738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caad929-6ccc-47ce-b28f-1085abbbd0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:32.447014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caad929-6ccc-47ce-b28f-1085abbbd0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:32.448645 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4caad929-6ccc-47ce-b28f-1085abbbd0b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:32.564501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caad929-6ccc-47ce-b28f-1085abbbd0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:32.565984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caad929-6ccc-47ce-b28f-1085abbbd0b3 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:32.586106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 654/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:34.853472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c74c65-0140-486f-81f5-0ab81224ea0d None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:34.854139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c74c65-0140-486f-81f5-0ab81224ea0d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:34.854139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c74c65-0140-486f-81f5-0ab81224ea0d None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:34.855326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 672/1326] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:34.860621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5b7e-e92b-4128-bd48-6d2a626d9448 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:34.861026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5b7e-e92b-4128-bd48-6d2a626d9448 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:34.861310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5b7e-e92b-4128-bd48-6d2a626d9448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:34.925523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c2ed5b7e-e92b-4128-bd48-6d2a626d9448 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:34.926758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c2ed5b7e-e92b-4128-bd48-6d2a626d9448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:35.021887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 655/1327] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:16:35.058296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c74c65-0140-486f-81f5-0ab81224ea0d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:35.059808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e5c74c65-0140-486f-81f5-0ab81224ea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:35.161752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c74c65-0140-486f-81f5-0ab81224ea0d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:35.233568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1f950-a84f-40f1-bd8a-938cd4f4e77b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:35.234253 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1f950-a84f-40f1-bd8a-938cd4f4e77b service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:35.234554 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1f950-a84f-40f1-bd8a-938cd4f4e77b service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:35.234818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1f950-a84f-40f1-bd8a-938cd4f4e77b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:35.236454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ffe1f950-a84f-40f1-bd8a-938cd4f4e77b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:35.341513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1f950-a84f-40f1-bd8a-938cd4f4e77b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:35.343005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1f950-a84f-40f1-bd8a-938cd4f4e77b service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:35.360162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 673/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:37.605283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318022-d356-4353-86e5-1ff8696c1380 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:37.605800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318022-d356-4353-86e5-1ff8696c1380 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:37.605800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318022-d356-4353-86e5-1ff8696c1380 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:37.607180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 656/1329] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:37.612411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858ed7-edfe-44fa-8d97-e8378b4e9525 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:37.612825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858ed7-edfe-44fa-8d97-e8378b4e9525 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:37.613107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858ed7-edfe-44fa-8d97-e8378b4e9525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:37.674521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d8858ed7-edfe-44fa-8d97-e8378b4e9525 None None] MFA Rules not processed for user `1ff1944c64d841c3a8596ed53714d48d`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:37.675638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d8858ed7-edfe-44fa-8d97-e8378b4e9525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:37.768243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 674/1330] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:37.805430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8318022-d356-4353-86e5-1ff8696c1380 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:37.807062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a8318022-d356-4353-86e5-1ff8696c1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:37.911733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8318022-d356-4353-86e5-1ff8696c1380 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:37.983378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6864ff-cf1b-4d42-a642-99d3183cb146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:37.984040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6864ff-cf1b-4d42-a642-99d3183cb146 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:37.984327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6864ff-cf1b-4d42-a642-99d3183cb146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:37.984607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6864ff-cf1b-4d42-a642-99d3183cb146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:37.986303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ee6864ff-cf1b-4d42-a642-99d3183cb146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:38.093341 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6864ff-cf1b-4d42-a642-99d3183cb146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:38.094822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6864ff-cf1b-4d42-a642-99d3183cb146 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:38.112136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 657/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:40.771200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da7f4-4295-463f-87d6-c2efb9502210 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:40.771750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da7f4-4295-463f-87d6-c2efb9502210 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:40.771893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da7f4-4295-463f-87d6-c2efb9502210 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:40.773717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 675/1332] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:40.779500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250113-abd0-45a9-b433-8e99816950e5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:40.779996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250113-abd0-45a9-b433-8e99816950e5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:40.780103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250113-abd0-45a9-b433-8e99816950e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:40.844854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-31250113-abd0-45a9-b433-8e99816950e5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:40.846022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-31250113-abd0-45a9-b433-8e99816950e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:40.939645 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 658/1333] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:40.975805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da7f4-4295-463f-87d6-c2efb9502210 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:40.977344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-928da7f4-4295-463f-87d6-c2efb9502210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:41.079027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da7f4-4295-463f-87d6-c2efb9502210 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:41.149596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c208256-1108-4727-bf43-e4c0bff0db06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:41.150193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208256-1108-4727-bf43-e4c0bff0db06 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:41.150472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208256-1108-4727-bf43-e4c0bff0db06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:41.150741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208256-1108-4727-bf43-e4c0bff0db06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:41.152349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8c208256-1108-4727-bf43-e4c0bff0db06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:41.258584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c208256-1108-4727-bf43-e4c0bff0db06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:41.260066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c208256-1108-4727-bf43-e4c0bff0db06 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:41.277512 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 676/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:43.578364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7987-64a7-46db-837d-65957170c436 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:43.578810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7987-64a7-46db-837d-65957170c436 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:43.578923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c7987-64a7-46db-837d-65957170c436 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:43.580324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 659/1335] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:43.585546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848b5c-60bb-40ff-89e9-7a5e6733e675 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:43.585987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848b5c-60bb-40ff-89e9-7a5e6733e675 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:43.586119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5848b5c-60bb-40ff-89e9-7a5e6733e675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:43.646902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e5848b5c-60bb-40ff-89e9-7a5e6733e675 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:43.648125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e5848b5c-60bb-40ff-89e9-7a5e6733e675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:43.737490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 677/1336] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:43.776785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7987-64a7-46db-837d-65957170c436 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:43.778387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e52c7987-64a7-46db-837d-65957170c436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:43.875446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c7987-64a7-46db-837d-65957170c436 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:43.946721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753793-093a-4806-93ec-e83e775ba2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:43.947637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753793-093a-4806-93ec-e83e775ba2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:43.948129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753793-093a-4806-93ec-e83e775ba2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:43.948706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753793-093a-4806-93ec-e83e775ba2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:43.951961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b753793-093a-4806-93ec-e83e775ba2f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:44.057365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753793-093a-4806-93ec-e83e775ba2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:44.058848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753793-093a-4806-93ec-e83e775ba2f0 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:44.077344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 660/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:47.055892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267ae3c-9966-489c-85b4-208585256149 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:47.056365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267ae3c-9966-489c-85b4-208585256149 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:47.056481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267ae3c-9966-489c-85b4-208585256149 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:47.057910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 678/1338] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:47.063132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147759-b210-4822-a63d-c5bc41485068 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:47.063570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147759-b210-4822-a63d-c5bc41485068 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:47.063698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147759-b210-4822-a63d-c5bc41485068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:47.127433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c9147759-b210-4822-a63d-c5bc41485068 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:47.128669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c9147759-b210-4822-a63d-c5bc41485068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:47.220104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 661/1339] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:47.255548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267ae3c-9966-489c-85b4-208585256149 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:47.257047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e267ae3c-9966-489c-85b4-208585256149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:47.356170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267ae3c-9966-489c-85b4-208585256149 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:47.426706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b928c17-b4b2-4b26-bea8-031cb8feccdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:47.427362 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928c17-b4b2-4b26-bea8-031cb8feccdf service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:47.427644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928c17-b4b2-4b26-bea8-031cb8feccdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:47.427917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928c17-b4b2-4b26-bea8-031cb8feccdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:47.429552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3b928c17-b4b2-4b26-bea8-031cb8feccdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:47.539465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b928c17-b4b2-4b26-bea8-031cb8feccdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:47.540918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b928c17-b4b2-4b26-bea8-031cb8feccdf service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:47.558080 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 679/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:52.064540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d9ae-ae34-4db1-8455-8e673087706d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:52.065090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d9ae-ae34-4db1-8455-8e673087706d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:52.065090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d9ae-ae34-4db1-8455-8e673087706d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:52.066517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 662/1341] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:52.073482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b7217-fcc0-4ff9-a1e5-92f49a117f1e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:52.074016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b7217-fcc0-4ff9-a1e5-92f49a117f1e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:52.074135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b7217-fcc0-4ff9-a1e5-92f49a117f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:52.141756 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-404b7217-fcc0-4ff9-a1e5-92f49a117f1e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:52.142976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-404b7217-fcc0-4ff9-a1e5-92f49a117f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:52.243494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 680/1342] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:52.281988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553d9ae-ae34-4db1-8455-8e673087706d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:52.284149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2553d9ae-ae34-4db1-8455-8e673087706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:52.375400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553d9ae-ae34-4db1-8455-8e673087706d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:52.378900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453e22a-c324-4999-bc3f-bed0e7dc680c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:52.379549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453e22a-c324-4999-bc3f-bed0e7dc680c service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:52.379840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453e22a-c324-4999-bc3f-bed0e7dc680c service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:52.380118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453e22a-c324-4999-bc3f-bed0e7dc680c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:52.388836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453e22a-c324-4999-bc3f-bed0e7dc680c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:52.390298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453e22a-c324-4999-bc3f-bed0e7dc680c service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:52.400111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 663/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:16:55.646987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b52f-404a-47e7-8e4b-ec4c39005ad0 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:55.647556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b52f-404a-47e7-8e4b-ec4c39005ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:55.647556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b52f-404a-47e7-8e4b-ec4c39005ad0 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:55.648893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 681/1344] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:16:55.654113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd2338-4a93-4336-becc-713036b7eb7b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:55.654518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd2338-4a93-4336-becc-713036b7eb7b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:55.654852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd2338-4a93-4336-becc-713036b7eb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:55.682623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-35cd2338-4a93-4336-becc-713036b7eb7b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:16:55.683822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-35cd2338-4a93-4336-becc-713036b7eb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:55.758117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 664/1345] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:16:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:16:55.794099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352b52f-404a-47e7-8e4b-ec4c39005ad0 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:16:55.795949 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5352b52f-404a-47e7-8e4b-ec4c39005ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:16:55.809173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352b52f-404a-47e7-8e4b-ec4c39005ad0 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:16:55.812671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154565c-ef48-4f9e-a999-f54ee54985dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:16:55.813306 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154565c-ef48-4f9e-a999-f54ee54985dd service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:16:55.813585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154565c-ef48-4f9e-a999-f54ee54985dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:16:55.813887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154565c-ef48-4f9e-a999-f54ee54985dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:16:55.823264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154565c-ef48-4f9e-a999-f54ee54985dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:16:55.824708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154565c-ef48-4f9e-a999-f54ee54985dd service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:16:55.834374 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 682/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:16:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:00.637504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe4fa9-26ec-4425-a5c8-39189c46d617 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:00.638051 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe4fa9-26ec-4425-a5c8-39189c46d617 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:00.638132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe4fa9-26ec-4425-a5c8-39189c46d617 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:00.639596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 665/1347] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:00.645145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a605e27-0183-4c43-8806-63cf97554d9d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:00.645605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a605e27-0183-4c43-8806-63cf97554d9d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:00.647537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a605e27-0183-4c43-8806-63cf97554d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:00.667778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6a605e27-0183-4c43-8806-63cf97554d9d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:00.668978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6a605e27-0183-4c43-8806-63cf97554d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:00.677010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 683/1348] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:00.715540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe4fa9-26ec-4425-a5c8-39189c46d617 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:00.726392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe4fa9-26ec-4425-a5c8-39189c46d617 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:00.729955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37809b2f-1ef2-4215-a622-ee7fe93bfa92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:00.730547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37809b2f-1ef2-4215-a622-ee7fe93bfa92 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:00.730831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37809b2f-1ef2-4215-a622-ee7fe93bfa92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:00.731106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37809b2f-1ef2-4215-a622-ee7fe93bfa92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:00.740284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37809b2f-1ef2-4215-a622-ee7fe93bfa92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:00.742270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37809b2f-1ef2-4215-a622-ee7fe93bfa92 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:00.751469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 666/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:06.025640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed2e87-1f54-4990-bd44-b4134ad64345 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:06.026562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed2e87-1f54-4990-bd44-b4134ad64345 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:06.026988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed2e87-1f54-4990-bd44-b4134ad64345 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:06.034553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 684/1350] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:17:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:06.047801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e2d01-4d82-43f6-90c7-dffb8f683955 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:06.048666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e2d01-4d82-43f6-90c7-dffb8f683955 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:06.049119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e2d01-4d82-43f6-90c7-dffb8f683955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:06.122618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ae4e2d01-4d82-43f6-90c7-dffb8f683955 None None] MFA Rules not processed for user `310c5d3ba81044ac8f3f192728cc3e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:06.124046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ae4e2d01-4d82-43f6-90c7-dffb8f683955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:06.199605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 667/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:17:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:10.645615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb690379-cf05-4a7c-a1f5-57d03005ca33 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:10.646715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb690379-cf05-4a7c-a1f5-57d03005ca33 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:10.647213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb690379-cf05-4a7c-a1f5-57d03005ca33 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:10.648684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 685/1352] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:10.655293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41497154-3ee2-4b7e-bf7b-1b165de05f7c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:10.655838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41497154-3ee2-4b7e-bf7b-1b165de05f7c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:10.656263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41497154-3ee2-4b7e-bf7b-1b165de05f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:10.675987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-41497154-3ee2-4b7e-bf7b-1b165de05f7c None None] MFA Rules not processed for user `310c5d3ba81044ac8f3f192728cc3e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:10.677392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-41497154-3ee2-4b7e-bf7b-1b165de05f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:10.691770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 668/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:17:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:12.191866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f930dd7-03a0-4d1b-80e9-74dbd36bac52 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:12.192395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f930dd7-03a0-4d1b-80e9-74dbd36bac52 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:12.192491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f930dd7-03a0-4d1b-80e9-74dbd36bac52 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:12.194627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 686/1354] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:12.199955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41653b0f-9125-4706-9d14-356a66bdff6a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:12.200423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41653b0f-9125-4706-9d14-356a66bdff6a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:12.200546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41653b0f-9125-4706-9d14-356a66bdff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:12.219104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-41653b0f-9125-4706-9d14-356a66bdff6a None None] MFA Rules not processed for user `310c5d3ba81044ac8f3f192728cc3e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:12.220306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-41653b0f-9125-4706-9d14-356a66bdff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:12.229133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 669/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:17:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:17:15.987613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daae522-aa44-4e3e-8caf-75167e33e1c8 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:15.988156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daae522-aa44-4e3e-8caf-75167e33e1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:15.988156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daae522-aa44-4e3e-8caf-75167e33e1c8 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:15.989844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 687/1356] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:15.994833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af976c2-65dd-4acd-bf6a-43defba462ef None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:15.995224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af976c2-65dd-4acd-bf6a-43defba462ef None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:15.995397 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af976c2-65dd-4acd-bf6a-43defba462ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:16.013420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6af976c2-65dd-4acd-bf6a-43defba462ef None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:16.014655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6af976c2-65dd-4acd-bf6a-43defba462ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:16.023323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 670/1357] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:16.071150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d8a01-c71f-48f9-9c84-6fc995a47581 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:16.071639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d8a01-c71f-48f9-9c84-6fc995a47581 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:16.072021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d8a01-c71f-48f9-9c84-6fc995a47581 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:16.073745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 688/1358] 10.222.0.25 () {54 vars in 961 bytes} [Fri May 13 13:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:16.079252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72c3da-a198-4bc1-8c01-54916c303b33 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:16.079796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72c3da-a198-4bc1-8c01-54916c303b33 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:16.080177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72c3da-a198-4bc1-8c01-54916c303b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:16.127210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bf72c3da-a198-4bc1-8c01-54916c303b33 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:16.128534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bf72c3da-a198-4bc1-8c01-54916c303b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:16.209386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 671/1359] 10.222.0.25 () {58 vars in 1058 bytes} [Fri May 13 13:17:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:16.217407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d8a01-c71f-48f9-9c84-6fc995a47581 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:16.235901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d8a01-c71f-48f9-9c84-6fc995a47581 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:16.241828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1f69e-a72b-475b-b127-fbd351bc128a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:16.242532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f69e-a72b-475b-b127-fbd351bc128a service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:16.242925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f69e-a72b-475b-b127-fbd351bc128a service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:16.243313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1f69e-a72b-475b-b127-fbd351bc128a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:16.252491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1f69e-a72b-475b-b127-fbd351bc128a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:16.254338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1f69e-a72b-475b-b127-fbd351bc128a service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:16.264689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 689/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:18.764625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9fa66-cbd0-4323-9e8c-1f26b905caf6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:18.765158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9fa66-cbd0-4323-9e8c-1f26b905caf6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:18.765357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9fa66-cbd0-4323-9e8c-1f26b905caf6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:18.767021 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 672/1361] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:18.772619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526bbc-ee49-4635-8558-373ce65affad None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:18.773044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526bbc-ee49-4635-8558-373ce65affad None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:18.773167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526bbc-ee49-4635-8558-373ce65affad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:18.791018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-68526bbc-ee49-4635-8558-373ce65affad None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:18.792208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-68526bbc-ee49-4635-8558-373ce65affad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:18.800116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 690/1362] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:18.848265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9fa66-cbd0-4323-9e8c-1f26b905caf6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:18.863724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9fa66-cbd0-4323-9e8c-1f26b905caf6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:18.872779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5bf37-be38-4b15-8f62-fb7f04739bf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:18.873434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5bf37-be38-4b15-8f62-fb7f04739bf5 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:18.873721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5bf37-be38-4b15-8f62-fb7f04739bf5 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:18.874023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5bf37-be38-4b15-8f62-fb7f04739bf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:18.883040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5bf37-be38-4b15-8f62-fb7f04739bf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:18.884795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5bf37-be38-4b15-8f62-fb7f04739bf5 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:18.895127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 673/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:21.560779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c7962-347c-44c0-90e5-c8677d801200 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:21.561325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c7962-347c-44c0-90e5-c8677d801200 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:21.561325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c7962-347c-44c0-90e5-c8677d801200 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:21.562979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 691/1364] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:21.569263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a7dcd-1f0f-4f0b-958e-b23d2e1ef99e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:21.569676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a7dcd-1f0f-4f0b-958e-b23d2e1ef99e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:21.571067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a7dcd-1f0f-4f0b-958e-b23d2e1ef99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:21.590709 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-be8a7dcd-1f0f-4f0b-958e-b23d2e1ef99e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:21.592367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-be8a7dcd-1f0f-4f0b-958e-b23d2e1ef99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:21.614454 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 674/1365] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:21.648092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9cabc-6892-4353-90b9-aebf1dd11814 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:21.648323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9cabc-6892-4353-90b9-aebf1dd11814 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:21.648599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9cabc-6892-4353-90b9-aebf1dd11814 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:21.649962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 692/1366] 10.222.0.25 () {54 vars in 961 bytes} [Fri May 13 13:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:21.655372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fe5a-962f-4e6f-9138-b7fb4601819d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:21.655755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fe5a-962f-4e6f-9138-b7fb4601819d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:21.656105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fe5a-962f-4e6f-9138-b7fb4601819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:21.674475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bfb0fe5a-962f-4e6f-9138-b7fb4601819d None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:21.675645 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bfb0fe5a-962f-4e6f-9138-b7fb4601819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:21.684779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 675/1367] 10.222.0.25 () {58 vars in 1058 bytes} [Fri May 13 13:17:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:21.692530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9cabc-6892-4353-90b9-aebf1dd11814 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:21.706807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9cabc-6892-4353-90b9-aebf1dd11814 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:21.712802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d7c75-59ed-4c74-a0b8-2add08565b33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:21.713467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d7c75-59ed-4c74-a0b8-2add08565b33 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:21.713752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d7c75-59ed-4c74-a0b8-2add08565b33 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:21.714068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d7c75-59ed-4c74-a0b8-2add08565b33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:21.722151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d7c75-59ed-4c74-a0b8-2add08565b33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:21.723751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d7c75-59ed-4c74-a0b8-2add08565b33 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:21.734387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 693/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:24.276405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cbf2b-4f19-4706-8e7f-43850a77f309 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:24.276995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cbf2b-4f19-4706-8e7f-43850a77f309 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:24.276995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cbf2b-4f19-4706-8e7f-43850a77f309 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:24.278480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 676/1369] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:24.284029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e1697-f67c-4948-b431-0faeb191cdba None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:24.284496 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e1697-f67c-4948-b431-0faeb191cdba None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:24.284586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e1697-f67c-4948-b431-0faeb191cdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:24.302745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-c38e1697-f67c-4948-b431-0faeb191cdba None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:24.302963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f468-435b-4982-9c16-b6c74985d58d None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:24.303061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f468-435b-4982-9c16-b6c74985d58d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:24.303296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f468-435b-4982-9c16-b6c74985d58d None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:24.303980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c38e1697-f67c-4948-b431-0faeb191cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:24.305021 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 677/1370] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:17:24.310018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713ff4-ba4a-4b1b-9bc5-884d1db6d421 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:24.310495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713ff4-ba4a-4b1b-9bc5-884d1db6d421 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:24.310779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713ff4-ba4a-4b1b-9bc5-884d1db6d421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:24.313004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 694/1371] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:24.329638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c0713ff4-ba4a-4b1b-9bc5-884d1db6d421 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:24.331094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c0713ff4-ba4a-4b1b-9bc5-884d1db6d421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:24.339902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 678/1372] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:17:24.359297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e1697-f67c-4948-b431-0faeb191cdba None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:24.372486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e1697-f67c-4948-b431-0faeb191cdba None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:24.378347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c106f73-04b8-41c0-9d11-a010433af516 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:24.379026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c106f73-04b8-41c0-9d11-a010433af516 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:24.379314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c106f73-04b8-41c0-9d11-a010433af516 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:24.380221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c106f73-04b8-41c0-9d11-a010433af516 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:24.390259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c106f73-04b8-41c0-9d11-a010433af516 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:24.391906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c106f73-04b8-41c0-9d11-a010433af516 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:24.403482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 695/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:24.665368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311b37f-a0cb-40f3-956b-f9d42f26ec7a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:24.665368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311b37f-a0cb-40f3-956b-f9d42f26ec7a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:24.665368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311b37f-a0cb-40f3-956b-f9d42f26ec7a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:24.667871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 679/1374] 10.222.0.25 () {54 vars in 963 bytes} [Fri May 13 13:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:24.672241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a52e3a-3c03-40ce-b9f8-a3d466bf72d2 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:24.672535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a52e3a-3c03-40ce-b9f8-a3d466bf72d2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:24.672817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a52e3a-3c03-40ce-b9f8-a3d466bf72d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:24.726835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-20a52e3a-3c03-40ce-b9f8-a3d466bf72d2 None None] MFA Rules not processed for user `a7955ca85d92485d91abfa401b63f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:24.728070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-20a52e3a-3c03-40ce-b9f8-a3d466bf72d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:24.796739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 696/1375] 10.222.0.25 () {58 vars in 1060 bytes} [Fri May 13 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:24.805346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311b37f-a0cb-40f3-956b-f9d42f26ec7a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:24.816523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311b37f-a0cb-40f3-956b-f9d42f26ec7a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:24.820605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d498d4-74b7-4171-85f4-0d1000d8b444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:24.821250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d498d4-74b7-4171-85f4-0d1000d8b444 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:24.821537 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d498d4-74b7-4171-85f4-0d1000d8b444 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:24.821841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d498d4-74b7-4171-85f4-0d1000d8b444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:24.831284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d498d4-74b7-4171-85f4-0d1000d8b444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:24.833041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d498d4-74b7-4171-85f4-0d1000d8b444 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:24.843241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 680/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:17:26.876215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7e72-3bf3-4767-ad4e-29a545dbf86b None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:26.876765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7e72-3bf3-4767-ad4e-29a545dbf86b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:26.876765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7e72-3bf3-4767-ad4e-29a545dbf86b None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:26.878238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 697/1377] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:26.884261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263e85-f8a9-46f8-8e85-83bbc549e2b0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:26.884712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263e85-f8a9-46f8-8e85-83bbc549e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:26.884809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb263e85-f8a9-46f8-8e85-83bbc549e2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:26.902378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cb263e85-f8a9-46f8-8e85-83bbc549e2b0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:26.903574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cb263e85-f8a9-46f8-8e85-83bbc549e2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:26.911437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 681/1378] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:26.956802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7e72-3bf3-4767-ad4e-29a545dbf86b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:26.971592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7e72-3bf3-4767-ad4e-29a545dbf86b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:26.977199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2b636-01cf-49cc-8930-62bf7edb80a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:26.977869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2b636-01cf-49cc-8930-62bf7edb80a2 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:26.978150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2b636-01cf-49cc-8930-62bf7edb80a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:26.978429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2b636-01cf-49cc-8930-62bf7edb80a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:26.987002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2b636-01cf-49cc-8930-62bf7edb80a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:26.988618 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2b636-01cf-49cc-8930-62bf7edb80a2 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:26.999228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 698/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:29.397337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c11180-7666-49b1-ab14-36684f6a1cd3 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:29.397935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c11180-7666-49b1-ab14-36684f6a1cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:29.397935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c11180-7666-49b1-ab14-36684f6a1cd3 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:29.399614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 682/1380] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:29.404794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054ce4-435e-463d-99c5-f3876caea113 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:29.405234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054ce4-435e-463d-99c5-f3876caea113 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:29.405362 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab054ce4-435e-463d-99c5-f3876caea113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:29.424351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ab054ce4-435e-463d-99c5-f3876caea113 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:29.425538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ab054ce4-435e-463d-99c5-f3876caea113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:29.434037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 699/1381] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:29.478540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c11180-7666-49b1-ab14-36684f6a1cd3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:29.498599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c11180-7666-49b1-ab14-36684f6a1cd3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:29.504336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77c920-4590-4423-aa57-145d6382d26b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:29.504979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77c920-4590-4423-aa57-145d6382d26b service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:29.505284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77c920-4590-4423-aa57-145d6382d26b service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:29.505568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77c920-4590-4423-aa57-145d6382d26b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:29.514485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77c920-4590-4423-aa57-145d6382d26b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:29.516129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77c920-4590-4423-aa57-145d6382d26b service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:29.525819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 683/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:31.943524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece0ad-c795-4612-8a09-94bd73994902 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:31.944498 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece0ad-c795-4612-8a09-94bd73994902 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:31.944498 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ece0ad-c795-4612-8a09-94bd73994902 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:31.945675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 700/1383] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:31.951426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08464-1321-411d-93b9-7fef51540437 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:31.951865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08464-1321-411d-93b9-7fef51540437 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:31.951990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08464-1321-411d-93b9-7fef51540437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:31.971078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a0e08464-1321-411d-93b9-7fef51540437 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:31.972332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a0e08464-1321-411d-93b9-7fef51540437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:31.980729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 684/1384] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:32.023013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ece0ad-c795-4612-8a09-94bd73994902 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:32.038247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ece0ad-c795-4612-8a09-94bd73994902 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:32.043872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f4d12-4f4e-4fd0-a936-6597345bc193 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:32.044636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f4d12-4f4e-4fd0-a936-6597345bc193 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:32.045019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f4d12-4f4e-4fd0-a936-6597345bc193 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:32.045493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f4d12-4f4e-4fd0-a936-6597345bc193 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:32.054742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f4d12-4f4e-4fd0-a936-6597345bc193 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:32.056406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f4d12-4f4e-4fd0-a936-6597345bc193 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:32.066651 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 701/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:34.341240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f259c-1888-4932-b61a-63f19f5d6f4f None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:34.341775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f259c-1888-4932-b61a-63f19f5d6f4f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:34.341894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f259c-1888-4932-b61a-63f19f5d6f4f None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:34.343732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 685/1386] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:34.350181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa19191-75e2-435b-95cc-a24e0e46815b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:34.350627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa19191-75e2-435b-95cc-a24e0e46815b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:34.350779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa19191-75e2-435b-95cc-a24e0e46815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:34.369269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6aa19191-75e2-435b-95cc-a24e0e46815b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:34.370503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6aa19191-75e2-435b-95cc-a24e0e46815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:34.378474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 702/1387] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:34.423428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f259c-1888-4932-b61a-63f19f5d6f4f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:34.438242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f259c-1888-4932-b61a-63f19f5d6f4f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:34.444576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a372572-d601-42fe-8de5-a99ae8e1eb2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:34.445217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372572-d601-42fe-8de5-a99ae8e1eb2c service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:34.445504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372572-d601-42fe-8de5-a99ae8e1eb2c service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:34.445807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372572-d601-42fe-8de5-a99ae8e1eb2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:34.454093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a372572-d601-42fe-8de5-a99ae8e1eb2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:34.455865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a372572-d601-42fe-8de5-a99ae8e1eb2c service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:34.466066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 686/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:36.863751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2543b08-9ff7-4a78-93af-884ea7e0f557 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:36.864273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2543b08-9ff7-4a78-93af-884ea7e0f557 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:36.864273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2543b08-9ff7-4a78-93af-884ea7e0f557 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:36.865699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 703/1389] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:36.871052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a69c6-44a0-475a-854c-ccbda21e8f11 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:36.871624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a69c6-44a0-475a-854c-ccbda21e8f11 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:36.871624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a69c6-44a0-475a-854c-ccbda21e8f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:36.890473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-de8a69c6-44a0-475a-854c-ccbda21e8f11 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:36.891678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-de8a69c6-44a0-475a-854c-ccbda21e8f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:36.900104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 687/1390] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:36.944690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2543b08-9ff7-4a78-93af-884ea7e0f557 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:36.961808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2543b08-9ff7-4a78-93af-884ea7e0f557 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:36.967365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414da37f-dbe8-419d-a9f4-968c40f41090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:36.968034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da37f-dbe8-419d-a9f4-968c40f41090 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:36.968321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da37f-dbe8-419d-a9f4-968c40f41090 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:36.968600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414da37f-dbe8-419d-a9f4-968c40f41090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:36.978498 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414da37f-dbe8-419d-a9f4-968c40f41090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:36.980516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414da37f-dbe8-419d-a9f4-968c40f41090 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:36.992307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 704/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:39.451493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34fa19-1285-4b6d-8d94-4212c7e540cc None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:39.452096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34fa19-1285-4b6d-8d94-4212c7e540cc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:39.452096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34fa19-1285-4b6d-8d94-4212c7e540cc None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:39.453568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 688/1392] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:39.459577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacc249-d746-400e-ab57-d901083960b0 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:39.460804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacc249-d746-400e-ab57-d901083960b0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:39.461241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacc249-d746-400e-ab57-d901083960b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:39.482043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-4aacc249-d746-400e-ab57-d901083960b0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:39.483292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4aacc249-d746-400e-ab57-d901083960b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:39.492153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 705/1393] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:39.536078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34fa19-1285-4b6d-8d94-4212c7e540cc None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:39.550568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34fa19-1285-4b6d-8d94-4212c7e540cc None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:39.556599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a18f1-a3da-4214-9eb9-ba160443f0e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:39.557829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a18f1-a3da-4214-9eb9-ba160443f0e4 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:39.557829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a18f1-a3da-4214-9eb9-ba160443f0e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:39.558044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a18f1-a3da-4214-9eb9-ba160443f0e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:39.566892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a18f1-a3da-4214-9eb9-ba160443f0e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:39.569840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a18f1-a3da-4214-9eb9-ba160443f0e4 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:39.579588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 689/1394] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:41.156270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0096c8e-e61d-47a5-8016-aae8155ffb5f None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:41.156838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0096c8e-e61d-47a5-8016-aae8155ffb5f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:41.156838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0096c8e-e61d-47a5-8016-aae8155ffb5f None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:41.158200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 706/1395] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:41.164006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643df88-05cb-45c6-9834-fbce63d1de56 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:41.164457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643df88-05cb-45c6-9834-fbce63d1de56 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:41.164573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643df88-05cb-45c6-9834-fbce63d1de56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:41.182255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5643df88-05cb-45c6-9834-fbce63d1de56 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:41.183370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5643df88-05cb-45c6-9834-fbce63d1de56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:41.191307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 690/1396] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:41.246231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d65242-a5b6-4a8e-ad54-b45b8d34e091 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:41.246520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d65242-a5b6-4a8e-ad54-b45b8d34e091 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:41.246797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d65242-a5b6-4a8e-ad54-b45b8d34e091 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:41.248134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 707/1397] 10.222.0.25 () {54 vars in 962 bytes} [Fri May 13 13:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:41.253089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9522a-11a8-47e3-b8f7-e830d97105a6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:41.253391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9522a-11a8-47e3-b8f7-e830d97105a6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:41.253674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9522a-11a8-47e3-b8f7-e830d97105a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:41.301157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2ee9522a-11a8-47e3-b8f7-e830d97105a6 None None] MFA Rules not processed for user `be3f786b791b4c8796d251948917ca93`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:41.302679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2ee9522a-11a8-47e3-b8f7-e830d97105a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:41.360544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 691/1398] 10.222.0.25 () {58 vars in 1059 bytes} [Fri May 13 13:17:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:41.368015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d65242-a5b6-4a8e-ad54-b45b8d34e091 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:41.378415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d65242-a5b6-4a8e-ad54-b45b8d34e091 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:41.382422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e936-129e-428c-98bc-15269df78321 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3f786b791b4c8796d251948917ca93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:41.383055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e936-129e-428c-98bc-15269df78321 service glance] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:41.383336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e936-129e-428c-98bc-15269df78321 service glance] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:41.383609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e936-129e-428c-98bc-15269df78321 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:41.392381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e936-129e-428c-98bc-15269df78321 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:41.393832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e936-129e-428c-98bc-15269df78321 service glance] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:41.403684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 708/1399] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:41.931437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b1b27-3ad0-481a-ac3d-66f7f76bd359 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:41.931751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b1b27-3ad0-481a-ac3d-66f7f76bd359 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:41.932035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b1b27-3ad0-481a-ac3d-66f7f76bd359 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:41.933487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 692/1400] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:41.938877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ca66-2c05-4f2f-ab92-47d2e86ce48c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:41.939166 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ca66-2c05-4f2f-ab92-47d2e86ce48c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:41.939438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ca66-2c05-4f2f-ab92-47d2e86ce48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:41.957396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-40d0ca66-2c05-4f2f-ab92-47d2e86ce48c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:41.959157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-40d0ca66-2c05-4f2f-ab92-47d2e86ce48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:41.967187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 709/1401] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:17:42.012192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b1b27-3ad0-481a-ac3d-66f7f76bd359 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:42.026732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b1b27-3ad0-481a-ac3d-66f7f76bd359 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:42.032058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe874f1-7dec-460d-90bc-7fc2f5d0da7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:42.032688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe874f1-7dec-460d-90bc-7fc2f5d0da7b service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:42.032974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe874f1-7dec-460d-90bc-7fc2f5d0da7b service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:42.033255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe874f1-7dec-460d-90bc-7fc2f5d0da7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:42.041976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe874f1-7dec-460d-90bc-7fc2f5d0da7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:42.043710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe874f1-7dec-460d-90bc-7fc2f5d0da7b service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:42.053458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 693/1402] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:17:42.151738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab604-bb2d-4e86-88fd-2b8ac514f441 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:42.163492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab604-bb2d-4e86-88fd-2b8ac514f441 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:42.163492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab604-bb2d-4e86-88fd-2b8ac514f441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:42.200841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-c9dab604-bb2d-4e86-88fd-2b8ac514f441 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:42.202079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c9dab604-bb2d-4e86-88fd-2b8ac514f441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:42.265982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 710/1403] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:17:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:42.309634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288ecd9-b330-450f-8c01-2826887c5950 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:42.310216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288ecd9-b330-450f-8c01-2826887c5950 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:42.310318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288ecd9-b330-450f-8c01-2826887c5950 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:42.311650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 694/1404] 10.222.0.25 () {54 vars in 942 bytes} [Fri May 13 13:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:42.319423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d24bcd-4c84-48d6-bd90-938cc246f14f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:42.319714 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d24bcd-4c84-48d6-bd90-938cc246f14f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:42.319992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d24bcd-4c84-48d6-bd90-938cc246f14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:42.371945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-14d24bcd-4c84-48d6-bd90-938cc246f14f None None] MFA Rules not processed for user `96f6d41c529444fba00bb09ac4224eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:42.373241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-14d24bcd-4c84-48d6-bd90-938cc246f14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:42.447772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 711/1405] 10.222.0.25 () {58 vars in 1039 bytes} [Fri May 13 13:17:42 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:42.457262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288ecd9-b330-450f-8c01-2826887c5950 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:42.471159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288ecd9-b330-450f-8c01-2826887c5950 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:42.476645 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158bf29-c247-4ec4-8a0d-66a74205beb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:42.477295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158bf29-c247-4ec4-8a0d-66a74205beb2 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:42.477578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158bf29-c247-4ec4-8a0d-66a74205beb2 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:42.477881 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158bf29-c247-4ec4-8a0d-66a74205beb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:42.486140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158bf29-c247-4ec4-8a0d-66a74205beb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:42.487724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158bf29-c247-4ec4-8a0d-66a74205beb2 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:42.496171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 695/1406] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:17:44.552837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35550a-cc0c-43a4-85e3-04bebcabfd3d None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:44.553478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35550a-cc0c-43a4-85e3-04bebcabfd3d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:44.553478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35550a-cc0c-43a4-85e3-04bebcabfd3d None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:44.554990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 712/1407] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:17:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:17:44.560646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420441a-b2ee-4d2c-9c40-f75ea7c6d351 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:44.561491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420441a-b2ee-4d2c-9c40-f75ea7c6d351 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:44.561800 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420441a-b2ee-4d2c-9c40-f75ea7c6d351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:44.581278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5420441a-b2ee-4d2c-9c40-f75ea7c6d351 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:17:44.582838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5420441a-b2ee-4d2c-9c40-f75ea7c6d351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:17:44.591892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 696/1408] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:17:44.652158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35550a-cc0c-43a4-85e3-04bebcabfd3d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:17:44.674118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35550a-cc0c-43a4-85e3-04bebcabfd3d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:17:44.680309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153d6d-f893-4558-868a-337854c1250d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:17:44.681036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153d6d-f893-4558-868a-337854c1250d service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:17:44.681398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153d6d-f893-4558-868a-337854c1250d service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:17:44.681815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153d6d-f893-4558-868a-337854c1250d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:17:44.691577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153d6d-f893-4558-868a-337854c1250d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:17:44.693437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153d6d-f893-4558-868a-337854c1250d service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:17:44.704296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 713/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:06.819460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0526398-96da-46fb-9cd8-3595668d4d42 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:06.820065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0526398-96da-46fb-9cd8-3595668d4d42 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:06.820065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0526398-96da-46fb-9cd8-3595668d4d42 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:06.821323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 697/1410] 10.222.0.25 () {54 vars in 918 bytes} [Fri May 13 13:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:06.826724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e1257-c312-41d0-ae66-3086315d087c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:06.827176 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e1257-c312-41d0-ae66-3086315d087c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:06.827279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e1257-c312-41d0-ae66-3086315d087c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:06.844712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-3d9e1257-c312-41d0-ae66-3086315d087c None None] MFA Rules not processed for user `310c5d3ba81044ac8f3f192728cc3e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:06.845942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3d9e1257-c312-41d0-ae66-3086315d087c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:06.855358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 714/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Fri May 13 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:06.869527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a553a-1dbe-4514-84e7-ef775908db7c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:06.869844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a553a-1dbe-4514-84e7-ef775908db7c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:06.870128 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a553a-1dbe-4514-84e7-ef775908db7c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:06.871467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 698/1412] 10.222.0.25 () {54 vars in 918 bytes} [Fri May 13 13:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:06.876053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e3ef-e8a9-4ac5-b24b-e90807e1c475 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:06.876347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e3ef-e8a9-4ac5-b24b-e90807e1c475 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:06.876626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e3ef-e8a9-4ac5-b24b-e90807e1c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:06.894472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6496e3ef-e8a9-4ac5-b24b-e90807e1c475 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:06.896423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6496e3ef-e8a9-4ac5-b24b-e90807e1c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:06.905955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 715/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Fri May 13 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:09.642631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb5f07-6cba-4bb0-9c1e-c2a143a8b72a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:09.643163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb5f07-6cba-4bb0-9c1e-c2a143a8b72a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:09.643276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb5f07-6cba-4bb0-9c1e-c2a143a8b72a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:09.644601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:09.650383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cacc99-c2b6-40ac-8356-e579264547ad None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:09.650806 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cacc99-c2b6-40ac-8356-e579264547ad None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:09.650957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cacc99-c2b6-40ac-8356-e579264547ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:09.669811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e0cacc99-c2b6-40ac-8356-e579264547ad None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:09.670931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e0cacc99-c2b6-40ac-8356-e579264547ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:09.678568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:09.720068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb5f07-6cba-4bb0-9c1e-c2a143a8b72a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:09.734666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb5f07-6cba-4bb0-9c1e-c2a143a8b72a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:09.742077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849c872-a181-4608-af2d-4ad1b49ce1ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:09.742722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849c872-a181-4608-af2d-4ad1b49ce1ff service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:09.743006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849c872-a181-4608-af2d-4ad1b49ce1ff service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:09.743283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849c872-a181-4608-af2d-4ad1b49ce1ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:09.751385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849c872-a181-4608-af2d-4ad1b49ce1ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:09.752965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849c872-a181-4608-af2d-4ad1b49ce1ff service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:09.762243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 700/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:10.757594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af509f-3286-470e-a9cd-e4e30f2d2de4 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:10.758150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af509f-3286-470e-a9cd-e4e30f2d2de4 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:10.758150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af509f-3286-470e-a9cd-e4e30f2d2de4 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:10.759463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:10.764366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4325c-1801-4f30-b6c5-5caf6b6c1cc9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:10.764807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4325c-1801-4f30-b6c5-5caf6b6c1cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:10.764954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4325c-1801-4f30-b6c5-5caf6b6c1cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:10.782276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d4b4325c-1801-4f30-b6c5-5caf6b6c1cc9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:10.783446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d4b4325c-1801-4f30-b6c5-5caf6b6c1cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:10.791582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:18:10.957772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af509f-3286-470e-a9cd-e4e30f2d2de4 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:10.974473 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af509f-3286-470e-a9cd-e4e30f2d2de4 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:10.979834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e648493-21df-4a53-9308-69ee96f66f69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:10.980476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648493-21df-4a53-9308-69ee96f66f69 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:10.980761 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648493-21df-4a53-9308-69ee96f66f69 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:10.981037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e648493-21df-4a53-9308-69ee96f66f69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:10.989202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e648493-21df-4a53-9308-69ee96f66f69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:10.990950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e648493-21df-4a53-9308-69ee96f66f69 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:11.000537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 718/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:11.954404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26648-c50c-46ed-9f7b-c9632f54ea76 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:11.954994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26648-c50c-46ed-9f7b-c9632f54ea76 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:11.954994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba26648-c50c-46ed-9f7b-c9632f54ea76 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:11.956339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:11.961848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e64ee3-c314-4be9-a1c9-5be38e9157db None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:11.962291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e64ee3-c314-4be9-a1c9-5be38e9157db None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:11.962431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e64ee3-c314-4be9-a1c9-5be38e9157db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:11.980502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-97e64ee3-c314-4be9-a1c9-5be38e9157db None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:11.981779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-97e64ee3-c314-4be9-a1c9-5be38e9157db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:11.989621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:12.141046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba26648-c50c-46ed-9f7b-c9632f54ea76 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:12.158421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba26648-c50c-46ed-9f7b-c9632f54ea76 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:12.163808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e892977-501f-4de2-8ec6-ce2659c0e22a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:12.164441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892977-501f-4de2-8ec6-ce2659c0e22a service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:12.164723 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892977-501f-4de2-8ec6-ce2659c0e22a service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:12.164999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e892977-501f-4de2-8ec6-ce2659c0e22a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:12.172904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e892977-501f-4de2-8ec6-ce2659c0e22a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:12.174510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e892977-501f-4de2-8ec6-ce2659c0e22a service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:12.183602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 703/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:13.139675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f776c-e047-44db-975f-0faf67a1b567 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:13.140255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f776c-e047-44db-975f-0faf67a1b567 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:13.140255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f776c-e047-44db-975f-0faf67a1b567 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:13.143826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:18:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:13.153002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebd8da-178c-4d32-b985-8d840b1799bd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:13.153452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebd8da-178c-4d32-b985-8d840b1799bd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:13.153651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebd8da-178c-4d32-b985-8d840b1799bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:13.173022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-88ebd8da-178c-4d32-b985-8d840b1799bd None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:13.174529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-88ebd8da-178c-4d32-b985-8d840b1799bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:13.182771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:13.342372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f776c-e047-44db-975f-0faf67a1b567 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:13.356514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f776c-e047-44db-975f-0faf67a1b567 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:13.361908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c7621-c54c-494f-a9b2-8c8fceffa0bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:13.362549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c7621-c54c-494f-a9b2-8c8fceffa0bf service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:13.362835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c7621-c54c-494f-a9b2-8c8fceffa0bf service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:13.363112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c7621-c54c-494f-a9b2-8c8fceffa0bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:13.373565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c7621-c54c-494f-a9b2-8c8fceffa0bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:13.375201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c7621-c54c-494f-a9b2-8c8fceffa0bf service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:13.385176 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 721/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:14.355848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468f224-d56f-4aae-90b3-de34e1bb9645 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:14.356434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468f224-d56f-4aae-90b3-de34e1bb9645 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:14.356434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468f224-d56f-4aae-90b3-de34e1bb9645 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:14.357870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:14.362712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec3f7f-3ea5-4421-98d1-fb5e2a183cac None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:14.363118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec3f7f-3ea5-4421-98d1-fb5e2a183cac None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:14.363289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec3f7f-3ea5-4421-98d1-fb5e2a183cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:14.380945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-8eec3f7f-3ea5-4421-98d1-fb5e2a183cac None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:14.382219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8eec3f7f-3ea5-4421-98d1-fb5e2a183cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:14.392226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:14.540719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468f224-d56f-4aae-90b3-de34e1bb9645 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:14.554714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468f224-d56f-4aae-90b3-de34e1bb9645 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:14.560041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f9e4e-ee02-4c5b-a70b-705ae02ea7b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:14.560667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9e4e-ee02-4c5b-a70b-705ae02ea7b9 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:14.560948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9e4e-ee02-4c5b-a70b-705ae02ea7b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:14.561224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9e4e-ee02-4c5b-a70b-705ae02ea7b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:14.569849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f9e4e-ee02-4c5b-a70b-705ae02ea7b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:14.571414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f9e4e-ee02-4c5b-a70b-705ae02ea7b9 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:14.580658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 706/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:15.529650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460a81f-0177-40fa-b08f-b9b08d49bafd None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:15.530161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460a81f-0177-40fa-b08f-b9b08d49bafd None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:15.530257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460a81f-0177-40fa-b08f-b9b08d49bafd None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:15.531629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:15.537178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc5b5-9151-4278-82b0-5454149c8a12 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:15.537609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc5b5-9151-4278-82b0-5454149c8a12 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:15.537716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fc5b5-9151-4278-82b0-5454149c8a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:15.555068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ba5fc5b5-9151-4278-82b0-5454149c8a12 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:15.556252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ba5fc5b5-9151-4278-82b0-5454149c8a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:15.563945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:15.713153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460a81f-0177-40fa-b08f-b9b08d49bafd None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:15.727886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460a81f-0177-40fa-b08f-b9b08d49bafd None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:15.733269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff318dbe-e27a-4fd2-81c1-c7f707e2d77b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:15.733937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff318dbe-e27a-4fd2-81c1-c7f707e2d77b service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:15.734227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff318dbe-e27a-4fd2-81c1-c7f707e2d77b service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:15.734508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff318dbe-e27a-4fd2-81c1-c7f707e2d77b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:15.742432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff318dbe-e27a-4fd2-81c1-c7f707e2d77b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:15.744025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff318dbe-e27a-4fd2-81c1-c7f707e2d77b service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:15.753105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 724/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:16.680688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b64b7-18a3-485d-a17d-2aa82ecc1960 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:16.681260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b64b7-18a3-485d-a17d-2aa82ecc1960 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:16.681260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b64b7-18a3-485d-a17d-2aa82ecc1960 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:16.682652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:16.687524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011389-593e-45fc-af37-ab6c9f524bcf None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:16.687961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011389-593e-45fc-af37-ab6c9f524bcf None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:16.688077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011389-593e-45fc-af37-ab6c9f524bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:16.706363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b4011389-593e-45fc-af37-ab6c9f524bcf None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:16.708523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b4011389-593e-45fc-af37-ab6c9f524bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:16.716526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:18:16.863444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b64b7-18a3-485d-a17d-2aa82ecc1960 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:16.877408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b64b7-18a3-485d-a17d-2aa82ecc1960 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:16.883564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f3415-0da7-4247-ba9b-ed31fec569ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:16.884202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3415-0da7-4247-ba9b-ed31fec569ea service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:16.884487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3415-0da7-4247-ba9b-ed31fec569ea service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:16.884765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f3415-0da7-4247-ba9b-ed31fec569ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:16.893096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f3415-0da7-4247-ba9b-ed31fec569ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:16.895312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f3415-0da7-4247-ba9b-ed31fec569ea service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:16.904680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 709/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:17.864753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779480f-f262-4fdb-bf3d-65071acf6368 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:17.865296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779480f-f262-4fdb-bf3d-65071acf6368 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:17.865296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779480f-f262-4fdb-bf3d-65071acf6368 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:17.866676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:17.871696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb5a7c-4450-4335-9ded-271184230e5f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:17.872092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb5a7c-4450-4335-9ded-271184230e5f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:17.872258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb5a7c-4450-4335-9ded-271184230e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:17.890833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a5fb5a7c-4450-4335-9ded-271184230e5f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:17.892057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a5fb5a7c-4450-4335-9ded-271184230e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:17.900398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:18.049005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779480f-f262-4fdb-bf3d-65071acf6368 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:18.064151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779480f-f262-4fdb-bf3d-65071acf6368 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:18.070980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af77a30-4517-414f-af5d-ce93a4273429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:18.071617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af77a30-4517-414f-af5d-ce93a4273429 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:18.071904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af77a30-4517-414f-af5d-ce93a4273429 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:18.072181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af77a30-4517-414f-af5d-ce93a4273429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:18.080882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af77a30-4517-414f-af5d-ce93a4273429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:18.082675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af77a30-4517-414f-af5d-ce93a4273429 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:18.092965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 727/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:19.042248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c5a35-57a1-4056-8038-ad667186ba2a None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:19.042827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c5a35-57a1-4056-8038-ad667186ba2a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:19.042827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c5a35-57a1-4056-8038-ad667186ba2a None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:19.044220 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:19.049146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba502927-c738-4ee9-b8b5-2155581f56c6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:19.049550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba502927-c738-4ee9-b8b5-2155581f56c6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:19.049697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba502927-c738-4ee9-b8b5-2155581f56c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:19.067598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ba502927-c738-4ee9-b8b5-2155581f56c6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:19.069835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ba502927-c738-4ee9-b8b5-2155581f56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:19.077424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:19.240713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c5a35-57a1-4056-8038-ad667186ba2a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:19.255242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c5a35-57a1-4056-8038-ad667186ba2a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:19.260589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cefd3f-c6a8-46e4-8da5-9233f1bd44fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:19.261226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cefd3f-c6a8-46e4-8da5-9233f1bd44fc service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:19.261513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cefd3f-c6a8-46e4-8da5-9233f1bd44fc service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:19.261814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cefd3f-c6a8-46e4-8da5-9233f1bd44fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:19.269692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cefd3f-c6a8-46e4-8da5-9233f1bd44fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:19.271303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cefd3f-c6a8-46e4-8da5-9233f1bd44fc service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:19.280625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 712/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:21.408840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c852-f512-44cd-93d5-00b280b403d7 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:21.409489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c852-f512-44cd-93d5-00b280b403d7 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:21.409489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c852-f512-44cd-93d5-00b280b403d7 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:21.411303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:21.416539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122ac91-1157-4f99-8a62-c390596c3362 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:21.416948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122ac91-1157-4f99-8a62-c390596c3362 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:21.417147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122ac91-1157-4f99-8a62-c390596c3362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:21.435441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0122ac91-1157-4f99-8a62-c390596c3362 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:21.436636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0122ac91-1157-4f99-8a62-c390596c3362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:21.444760 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:21.658092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c852-f512-44cd-93d5-00b280b403d7 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:21.658092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c852-f512-44cd-93d5-00b280b403d7 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:21.658092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ace7b-caee-42bf-bfb9-289d68cc45cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:21.658092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ace7b-caee-42bf-bfb9-289d68cc45cc service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:21.658795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ace7b-caee-42bf-bfb9-289d68cc45cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:21.658795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ace7b-caee-42bf-bfb9-289d68cc45cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:21.658795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ace7b-caee-42bf-bfb9-289d68cc45cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:21.658795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ace7b-caee-42bf-bfb9-289d68cc45cc service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:21.658795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 730/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:24.002052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bbc9f-591d-41be-a84c-35c6898c4609 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:24.002652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bbc9f-591d-41be-a84c-35c6898c4609 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:24.002652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bbc9f-591d-41be-a84c-35c6898c4609 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:24.004151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:24.009466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc750dff-bb07-4bde-ad43-8017b620c8e6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:24.009908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc750dff-bb07-4bde-ad43-8017b620c8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:24.010077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc750dff-bb07-4bde-ad43-8017b620c8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:24.027781 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-dc750dff-bb07-4bde-ad43-8017b620c8e6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:24.028994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-dc750dff-bb07-4bde-ad43-8017b620c8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:24.037251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:18:24.074968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bbc9f-591d-41be-a84c-35c6898c4609 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:24.086131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bbc9f-591d-41be-a84c-35c6898c4609 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:24.089577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5220b-1b70-40f8-aa97-3b16fa9129c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:24.090247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5220b-1b70-40f8-aa97-3b16fa9129c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:24.090532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5220b-1b70-40f8-aa97-3b16fa9129c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:24.090803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5220b-1b70-40f8-aa97-3b16fa9129c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:24.099468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5220b-1b70-40f8-aa97-3b16fa9129c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:24.100908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5220b-1b70-40f8-aa97-3b16fa9129c6 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:24.110742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 715/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:26.681623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9917-6c61-4d19-b1b0-6b1ab0b4258a None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:26.682264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9917-6c61-4d19-b1b0-6b1ab0b4258a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:26.682264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9917-6c61-4d19-b1b0-6b1ab0b4258a None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:26.683574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:26.689129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d144-0e18-4c48-95d2-4102b78add16 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:26.689605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d144-0e18-4c48-95d2-4102b78add16 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:26.689715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d144-0e18-4c48-95d2-4102b78add16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:26.707647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5bb3d144-0e18-4c48-95d2-4102b78add16 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:26.708838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5bb3d144-0e18-4c48-95d2-4102b78add16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:26.717605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:26.755948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea9917-6c61-4d19-b1b0-6b1ab0b4258a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:26.766396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea9917-6c61-4d19-b1b0-6b1ab0b4258a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:26.770354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c77d94-301d-4b8f-8c26-80f636f04964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:26.774658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c77d94-301d-4b8f-8c26-80f636f04964 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:26.774658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c77d94-301d-4b8f-8c26-80f636f04964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:26.774658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c77d94-301d-4b8f-8c26-80f636f04964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:26.780543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c77d94-301d-4b8f-8c26-80f636f04964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:26.782017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c77d94-301d-4b8f-8c26-80f636f04964 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:26.791959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 733/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:29.111020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341dbf3-03a8-47c3-aadb-9bd3e5bcd089 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:29.111604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341dbf3-03a8-47c3-aadb-9bd3e5bcd089 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:29.111604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341dbf3-03a8-47c3-aadb-9bd3e5bcd089 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:29.113474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:29.118978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b12dd-f3f5-4baa-967a-d5e81bb3d6f7 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:29.119400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b12dd-f3f5-4baa-967a-d5e81bb3d6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:29.119543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b12dd-f3f5-4baa-967a-d5e81bb3d6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:29.137371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-511b12dd-f3f5-4baa-967a-d5e81bb3d6f7 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:29.138601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-511b12dd-f3f5-4baa-967a-d5e81bb3d6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:29.147020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:29.184494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341dbf3-03a8-47c3-aadb-9bd3e5bcd089 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:29.198666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341dbf3-03a8-47c3-aadb-9bd3e5bcd089 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:29.202240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63603674-6b59-41a2-8b16-17e51741b686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:29.202896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63603674-6b59-41a2-8b16-17e51741b686 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:29.203179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63603674-6b59-41a2-8b16-17e51741b686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:29.203453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63603674-6b59-41a2-8b16-17e51741b686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:29.214046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63603674-6b59-41a2-8b16-17e51741b686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:29.215490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63603674-6b59-41a2-8b16-17e51741b686 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:29.225652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 718/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:31.465962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf0f3-ea85-4ff5-94a6-94d222931a4a None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:31.466467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf0f3-ea85-4ff5-94a6-94d222931a4a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:31.466564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf0f3-ea85-4ff5-94a6-94d222931a4a None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:31.467880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:31.473190 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d84b2-c149-4dba-8efc-0f2f194f7bd3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:31.473589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d84b2-c149-4dba-8efc-0f2f194f7bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:31.473744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d84b2-c149-4dba-8efc-0f2f194f7bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:31.492641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-981d84b2-c149-4dba-8efc-0f2f194f7bd3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:31.493855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-981d84b2-c149-4dba-8efc-0f2f194f7bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:31.501718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:31.539254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cf0f3-ea85-4ff5-94a6-94d222931a4a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:31.549961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cf0f3-ea85-4ff5-94a6-94d222931a4a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:31.553985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941cf92-577c-4830-ae8b-f7d7b11399d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:31.554614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941cf92-577c-4830-ae8b-f7d7b11399d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:31.554895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941cf92-577c-4830-ae8b-f7d7b11399d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:31.555171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941cf92-577c-4830-ae8b-f7d7b11399d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:31.563647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941cf92-577c-4830-ae8b-f7d7b11399d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:31.565068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941cf92-577c-4830-ae8b-f7d7b11399d6 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:31.574854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 736/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:33.838482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c8b9-3ce8-485f-8588-8feb882ef369 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:33.839056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c8b9-3ce8-485f-8588-8feb882ef369 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:33.839056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51c8b9-3ce8-485f-8588-8feb882ef369 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:33.840409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:33.845950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830052-cefe-4a3f-a1aa-4f0465672fa5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:33.846361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830052-cefe-4a3f-a1aa-4f0465672fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:33.846525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830052-cefe-4a3f-a1aa-4f0465672fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:33.864875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b4830052-cefe-4a3f-a1aa-4f0465672fa5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:33.866131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b4830052-cefe-4a3f-a1aa-4f0465672fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:33.873838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:33.912631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51c8b9-3ce8-485f-8588-8feb882ef369 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:33.924746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51c8b9-3ce8-485f-8588-8feb882ef369 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:33.928948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159461a-2cbb-4fe5-a7d1-4f1b49e94e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:33.929591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159461a-2cbb-4fe5-a7d1-4f1b49e94e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:33.929908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159461a-2cbb-4fe5-a7d1-4f1b49e94e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:33.930188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159461a-2cbb-4fe5-a7d1-4f1b49e94e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:33.939240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159461a-2cbb-4fe5-a7d1-4f1b49e94e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:33.940660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159461a-2cbb-4fe5-a7d1-4f1b49e94e2b service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:33.950110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 721/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:36.216140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408683a5-405c-4d90-8861-2b1ebd659b4b None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:36.216708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408683a5-405c-4d90-8861-2b1ebd659b4b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:36.216708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408683a5-405c-4d90-8861-2b1ebd659b4b None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:36.218144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:36.223439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259604f1-d51e-4fc6-9c09-f267932b48cb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:36.223872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259604f1-d51e-4fc6-9c09-f267932b48cb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:36.223998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259604f1-d51e-4fc6-9c09-f267932b48cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:36.241288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-259604f1-d51e-4fc6-9c09-f267932b48cb None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:36.242619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-259604f1-d51e-4fc6-9c09-f267932b48cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:36.250342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:36.290726 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408683a5-405c-4d90-8861-2b1ebd659b4b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:36.303747 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408683a5-405c-4d90-8861-2b1ebd659b4b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:36.307362 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e7718-0cbc-4743-83fd-318122f81543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:36.312124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7718-0cbc-4743-83fd-318122f81543 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:36.312124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7718-0cbc-4743-83fd-318122f81543 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:36.312124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7718-0cbc-4743-83fd-318122f81543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:36.318378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e7718-0cbc-4743-83fd-318122f81543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:36.320081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e7718-0cbc-4743-83fd-318122f81543 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:36.330784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 739/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:38.620957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa10ba-5850-4ddc-9490-e31dba496408 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:38.621525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa10ba-5850-4ddc-9490-e31dba496408 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:38.621525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa10ba-5850-4ddc-9490-e31dba496408 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:38.622918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:38.628668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63eb7-fb85-4054-a8d1-d89965b7f8a8 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:38.629064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63eb7-fb85-4054-a8d1-d89965b7f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:38.629248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63eb7-fb85-4054-a8d1-d89965b7f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:38.646823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d5e63eb7-fb85-4054-a8d1-d89965b7f8a8 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:38.648004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d5e63eb7-fb85-4054-a8d1-d89965b7f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:38.655696 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:38.691922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa10ba-5850-4ddc-9490-e31dba496408 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:38.702630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa10ba-5850-4ddc-9490-e31dba496408 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:38.706136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da721624-f945-46ca-a858-4f35a3b65808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:38.707316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721624-f945-46ca-a858-4f35a3b65808 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:38.707595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721624-f945-46ca-a858-4f35a3b65808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:38.707864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da721624-f945-46ca-a858-4f35a3b65808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:38.716447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da721624-f945-46ca-a858-4f35a3b65808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:38.717894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da721624-f945-46ca-a858-4f35a3b65808 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:38.727002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 724/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:18:40.938412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afb39a-c776-4139-a8d2-ac775ed05623 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:40.938929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afb39a-c776-4139-a8d2-ac775ed05623 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:40.938929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afb39a-c776-4139-a8d2-ac775ed05623 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:40.940252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:40.946441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47069352-4f7d-4c2f-a345-7f799642ecab None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:40.946893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47069352-4f7d-4c2f-a345-7f799642ecab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:40.947002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47069352-4f7d-4c2f-a345-7f799642ecab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:40.964893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-47069352-4f7d-4c2f-a345-7f799642ecab None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:40.966111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-47069352-4f7d-4c2f-a345-7f799642ecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:40.974081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:41.009733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afb39a-c776-4139-a8d2-ac775ed05623 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:41.020118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afb39a-c776-4139-a8d2-ac775ed05623 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:41.023600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04847f4-bc9e-47c2-a767-34b30bd5f91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:41.024233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04847f4-bc9e-47c2-a767-34b30bd5f91e service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:41.024518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04847f4-bc9e-47c2-a767-34b30bd5f91e service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:41.024799 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04847f4-bc9e-47c2-a767-34b30bd5f91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:41.033484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04847f4-bc9e-47c2-a767-34b30bd5f91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:41.034986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04847f4-bc9e-47c2-a767-34b30bd5f91e service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:41.044134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 742/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:18:43.260365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8cf44-a6f7-433b-ac50-7d1039c99f44 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:43.260934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8cf44-a6f7-433b-ac50-7d1039c99f44 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:43.260934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8cf44-a6f7-433b-ac50-7d1039c99f44 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:43.263039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:18:43.268407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00947e-90e8-44c1-900f-562f26584300 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:43.269313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00947e-90e8-44c1-900f-562f26584300 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:43.269509 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00947e-90e8-44c1-900f-562f26584300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:43.287127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-dc00947e-90e8-44c1-900f-562f26584300 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:18:43.288313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-dc00947e-90e8-44c1-900f-562f26584300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:18:43.296035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:18:43.331822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8cf44-a6f7-433b-ac50-7d1039c99f44 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:18:43.342229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8cf44-a6f7-433b-ac50-7d1039c99f44 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:18:43.346048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d7d1-518d-4cf4-bab3-66e8dd27bbca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:18:43.346680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d7d1-518d-4cf4-bab3-66e8dd27bbca service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:18:43.347527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d7d1-518d-4cf4-bab3-66e8dd27bbca service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:18:43.347797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d7d1-518d-4cf4-bab3-66e8dd27bbca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:18:43.356459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d7d1-518d-4cf4-bab3-66e8dd27bbca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:18:43.357915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d7d1-518d-4cf4-bab3-66e8dd27bbca service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:18:43.367615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 727/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:19:16.864336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a719a-4d7f-4ef7-bbce-5670f5d7c773 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:16.864898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a719a-4d7f-4ef7-bbce-5670f5d7c773 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:16.864898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a719a-4d7f-4ef7-bbce-5670f5d7c773 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:16.866223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 744/1471] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:19:16.871389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61e749-1d63-40ff-ae62-53f48cf88294 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:16.871805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61e749-1d63-40ff-ae62-53f48cf88294 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:16.871944 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61e749-1d63-40ff-ae62-53f48cf88294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:16.888993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2e61e749-1d63-40ff-ae62-53f48cf88294 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:19:16.891363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2e61e749-1d63-40ff-ae62-53f48cf88294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:19:16.899218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 728/1472] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:19:16.946208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa5796-cc71-488d-ab0b-0f7bda0869d2 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:16.950935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa5796-cc71-488d-ab0b-0f7bda0869d2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:16.950935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa5796-cc71-488d-ab0b-0f7bda0869d2 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:16.950935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 745/1473] 10.222.0.25 () {54 vars in 962 bytes} [Fri May 13 13:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:19:16.952708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bf300-7263-462c-ae7c-e1bc1a1bb8db None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:16.953000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bf300-7263-462c-ae7c-e1bc1a1bb8db None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:16.953274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bf300-7263-462c-ae7c-e1bc1a1bb8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:16.973083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-509bf300-7263-462c-ae7c-e1bc1a1bb8db None None] MFA Rules not processed for user `be3f786b791b4c8796d251948917ca93`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:19:16.974365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-509bf300-7263-462c-ae7c-e1bc1a1bb8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:19:16.980810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 729/1474] 10.222.0.25 () {58 vars in 1059 bytes} [Fri May 13 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:19:16.988121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa5796-cc71-488d-ab0b-0f7bda0869d2 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:19:16.998882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa5796-cc71-488d-ab0b-0f7bda0869d2 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:19:17.002626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c210b00-0a8a-4d21-85b5-35d1fefbcd34 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3f786b791b4c8796d251948917ca93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:19:17.003281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c210b00-0a8a-4d21-85b5-35d1fefbcd34 service glance] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:17.003563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c210b00-0a8a-4d21-85b5-35d1fefbcd34 service glance] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:17.003834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c210b00-0a8a-4d21-85b5-35d1fefbcd34 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:17.012691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c210b00-0a8a-4d21-85b5-35d1fefbcd34 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:19:17.014139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c210b00-0a8a-4d21-85b5-35d1fefbcd34 service glance] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:19:17.023605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 746/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:19:19.340302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa4f30-79c0-43dd-8d0f-b8e193fc12eb None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:19.340831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa4f30-79c0-43dd-8d0f-b8e193fc12eb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:19.340935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa4f30-79c0-43dd-8d0f-b8e193fc12eb None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:19.342504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 730/1476] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:19:19.347891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb2ad0-8033-4da4-b66a-404b9abe1b66 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:19.348341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb2ad0-8033-4da4-b66a-404b9abe1b66 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:19.348454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb2ad0-8033-4da4-b66a-404b9abe1b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:19.365711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d0cb2ad0-8033-4da4-b66a-404b9abe1b66 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:19:19.366969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d0cb2ad0-8033-4da4-b66a-404b9abe1b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:19:19.374685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 747/1477] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:19:19.398519 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa4f30-79c0-43dd-8d0f-b8e193fc12eb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:19:19.410973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa4f30-79c0-43dd-8d0f-b8e193fc12eb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:19:19.415530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9435-11e1-46b8-a199-3ac2b5408d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:19:19.416196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9435-11e1-46b8-a199-3ac2b5408d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:19.416545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9435-11e1-46b8-a199-3ac2b5408d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:19.416825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9435-11e1-46b8-a199-3ac2b5408d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:19.417564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf9435-11e1-46b8-a199-3ac2b5408d12 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:19:19.419368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-d4cf9435-11e1-46b8-a199-3ac2b5408d12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:19:19.419368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: warnings.warn(msg) May 13 13:19:19.419368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: May 13 13:19:19.420522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf9435-11e1-46b8-a199-3ac2b5408d12 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:19:19.440193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 731/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Fri May 13 13:19:19 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:19:21.679952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b2f1b-2d2f-4cbf-bd4e-251fc7d84bbe None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:21.680541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b2f1b-2d2f-4cbf-bd4e-251fc7d84bbe None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:21.680541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b2f1b-2d2f-4cbf-bd4e-251fc7d84bbe None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:21.681898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 748/1479] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:19:21.687229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd7834-985c-472f-bd37-20b95dba380d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:21.687624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd7834-985c-472f-bd37-20b95dba380d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:21.687792 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd7834-985c-472f-bd37-20b95dba380d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:21.706542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e0dd7834-985c-472f-bd37-20b95dba380d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:19:21.707692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e0dd7834-985c-472f-bd37-20b95dba380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:19:21.715625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 732/1480] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:19:21.758202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b2f1b-2d2f-4cbf-bd4e-251fc7d84bbe None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:19:21.773144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b2f1b-2d2f-4cbf-bd4e-251fc7d84bbe None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:19:21.778782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e85474-3d61-4000-9d08-294969f5b983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:19:21.779427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e85474-3d61-4000-9d08-294969f5b983 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:21.779713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e85474-3d61-4000-9d08-294969f5b983 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:21.779991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e85474-3d61-4000-9d08-294969f5b983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:21.789067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e85474-3d61-4000-9d08-294969f5b983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:19:21.790808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e85474-3d61-4000-9d08-294969f5b983 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:19:21.800649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 749/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:19:24.201487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffac4be-e0f3-44fd-8c01-897ae3af44a6 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:24.202077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffac4be-e0f3-44fd-8c01-897ae3af44a6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:24.202077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffac4be-e0f3-44fd-8c01-897ae3af44a6 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:24.203457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 733/1482] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:19:24.208738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec16416-ff98-4544-86dd-8c98b4bcf1dc None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:24.209188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec16416-ff98-4544-86dd-8c98b4bcf1dc None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:24.209315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec16416-ff98-4544-86dd-8c98b4bcf1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:24.227030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-4ec16416-ff98-4544-86dd-8c98b4bcf1dc None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:19:24.228237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4ec16416-ff98-4544-86dd-8c98b4bcf1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:19:24.235951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 750/1483] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:19:24.271542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffac4be-e0f3-44fd-8c01-897ae3af44a6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:19:24.282068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffac4be-e0f3-44fd-8c01-897ae3af44a6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:19:24.286140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2024db-be04-4410-abfc-1ee02b066aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:19:24.286776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2024db-be04-4410-abfc-1ee02b066aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:24.287060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2024db-be04-4410-abfc-1ee02b066aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:24.287340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2024db-be04-4410-abfc-1ee02b066aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:24.296010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2024db-be04-4410-abfc-1ee02b066aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:19:24.297436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2024db-be04-4410-abfc-1ee02b066aa0 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:19:24.307141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 734/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:19:26.565416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfcfa-2dde-4924-b276-00c63323839d None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:26.565987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfcfa-2dde-4924-b276-00c63323839d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:26.566087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfcfa-2dde-4924-b276-00c63323839d None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:26.567426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 751/1485] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:19:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:19:26.572963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bcbab-0fb0-401b-ae06-526f20d04401 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:26.573368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bcbab-0fb0-401b-ae06-526f20d04401 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:26.573512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bcbab-0fb0-401b-ae06-526f20d04401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:26.591231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a90bcbab-0fb0-401b-ae06-526f20d04401 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:19:26.592431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a90bcbab-0fb0-401b-ae06-526f20d04401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:19:26.600043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 735/1486] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:19:26.637768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bfcfa-2dde-4924-b276-00c63323839d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:19:26.648722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bfcfa-2dde-4924-b276-00c63323839d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:19:26.652257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e91c5-03e9-4a22-8d91-b561a186f12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:19:26.652887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e91c5-03e9-4a22-8d91-b561a186f12b service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:26.653175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e91c5-03e9-4a22-8d91-b561a186f12b service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:26.653456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e91c5-03e9-4a22-8d91-b561a186f12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:26.662189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e91c5-03e9-4a22-8d91-b561a186f12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:19:26.663621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e91c5-03e9-4a22-8d91-b561a186f12b service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:19:26.672737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 752/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:19:29.104872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2828-ef5e-4bdb-a315-e79fbcbb21f0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:29.105488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2828-ef5e-4bdb-a315-e79fbcbb21f0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:29.105597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2828-ef5e-4bdb-a315-e79fbcbb21f0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:29.106984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 736/1488] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:19:29.112294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097c4b2-73d7-4c15-869b-ecd2b0075315 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:29.112730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097c4b2-73d7-4c15-869b-ecd2b0075315 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:29.112895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097c4b2-73d7-4c15-869b-ecd2b0075315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:29.130647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0097c4b2-73d7-4c15-869b-ecd2b0075315 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:19:29.131839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0097c4b2-73d7-4c15-869b-ecd2b0075315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:19:29.139643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 753/1489] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:19:29.179596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a2828-ef5e-4bdb-a315-e79fbcbb21f0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:19:29.190041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a2828-ef5e-4bdb-a315-e79fbcbb21f0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:19:29.193507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9929b95-4ad9-412a-b79f-e596216a290d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:19:29.194181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9929b95-4ad9-412a-b79f-e596216a290d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:29.194718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9929b95-4ad9-412a-b79f-e596216a290d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:29.195007 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9929b95-4ad9-412a-b79f-e596216a290d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:29.203647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9929b95-4ad9-412a-b79f-e596216a290d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:19:29.205568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9929b95-4ad9-412a-b79f-e596216a290d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:19:29.214797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 737/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:19:31.807267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443673aa-906e-44b0-b00d-b607b63307ef None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:31.807832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443673aa-906e-44b0-b00d-b607b63307ef None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:31.807832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443673aa-906e-44b0-b00d-b607b63307ef None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:31.809190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 754/1491] 10.222.0.25 () {54 vars in 926 bytes} [Fri May 13 13:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:19:31.814592 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea362c9-b727-42e4-be4f-2cc5fb7c5f0b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:31.815066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea362c9-b727-42e4-be4f-2cc5fb7c5f0b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:31.815182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea362c9-b727-42e4-be4f-2cc5fb7c5f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:31.832636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cea362c9-b727-42e4-be4f-2cc5fb7c5f0b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:19:31.833840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cea362c9-b727-42e4-be4f-2cc5fb7c5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:19:31.841514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 738/1492] 10.222.0.25 () {58 vars in 1023 bytes} [Fri May 13 13:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:19:31.880489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443673aa-906e-44b0-b00d-b607b63307ef None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:19:31.890925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443673aa-906e-44b0-b00d-b607b63307ef None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:19:31.894452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d6f8b-77ab-4e9f-8c99-d7ce68893ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:19:31.895079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d6f8b-77ab-4e9f-8c99-d7ce68893ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:19:31.895360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d6f8b-77ab-4e9f-8c99-d7ce68893ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:19:31.895632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d6f8b-77ab-4e9f-8c99-d7ce68893ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:19:31.904218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d6f8b-77ab-4e9f-8c99-d7ce68893ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:19:31.905647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d6f8b-77ab-4e9f-8c99-d7ce68893ef9 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:19:31.915396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 755/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:21:18.570575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99eca0d-6526-42d9-8e62-0561314d3e63 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:18.571144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99eca0d-6526-42d9-8e62-0561314d3e63 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:18.571144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99eca0d-6526-42d9-8e62-0561314d3e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:18.589238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e99eca0d-6526-42d9-8e62-0561314d3e63 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:21:18.590448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e99eca0d-6526-42d9-8e62-0561314d3e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:21:18.598132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 739/1494] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:21:18.607207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967c28c-de01-491f-b0f2-6f8785559bdb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:18.607699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967c28c-de01-491f-b0f2-6f8785559bdb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:18.607804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967c28c-de01-491f-b0f2-6f8785559bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:18.625769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-9967c28c-de01-491f-b0f2-6f8785559bdb None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:21:18.627055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9967c28c-de01-491f-b0f2-6f8785559bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:21:18.634822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 756/1495] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:21:18.639230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99eca0d-6526-42d9-8e62-0561314d3e63 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:18.649365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99eca0d-6526-42d9-8e62-0561314d3e63 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:18.654056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:18.654713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:18.655000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:18.655282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:18.656039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:21:18.656347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:18.657355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 13:21:18.657355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: warnings.warn(msg) May 13 13:21:18.657355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: May 13 13:21:18.658291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:18.672262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 740/1496] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:21:18.680407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967c28c-de01-491f-b0f2-6f8785559bdb None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:18.690054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967c28c-de01-491f-b0f2-6f8785559bdb None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:18.694592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:18.695491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:18.695767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:18.696038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:18.696724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:18.698533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WARNING py.warnings [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:21:18.698533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: warnings.warn(msg) May 13 13:21:18.698533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: May 13 13:21:18.699718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:18.830524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 757/1497] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:21:18 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:21:18.834591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:18.844326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf4a9b-dd67-4a59-8546-7b1a18d62872 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:18.898240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:18.898905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:18.899184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:18.899745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:18.900460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:18.901484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:21:18.901484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: warnings.warn(msg) May 13 13:21:18.901484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: May 13 13:21:18.902725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:18.981814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 741/1498] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:21:18 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:21:18.986577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:18.996692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208c747-430d-4b0d-8f1e-7be23b3895ad admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:19.001741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:19.002436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.008099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.008099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.008099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:19.008099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WARNING py.warnings [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 13:21:19.008099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: warnings.warn(msg) May 13 13:21:19.008099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: May 13 13:21:19.008099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:19.017865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 758/1499] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:21:19.021909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:19.031566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2df8a4-9033-4cc0-b80a-0e5413422a55 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:19.036116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312932b-614e-42e9-8fb6-5eea7e529acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:19.036767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312932b-614e-42e9-8fb6-5eea7e529acb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.037055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312932b-614e-42e9-8fb6-5eea7e529acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.037336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312932b-614e-42e9-8fb6-5eea7e529acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.038095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312932b-614e-42e9-8fb6-5eea7e529acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:19.039101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-d312932b-614e-42e9-8fb6-5eea7e529acb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 13:21:19.039101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: warnings.warn(msg) May 13 13:21:19.039101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: May 13 13:21:19.039995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312932b-614e-42e9-8fb6-5eea7e529acb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:19.053368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 742/1500] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:21:19.057336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:19.067584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e05de-f1c8-4a40-8c6b-5742ea3f11af admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:19.072295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15ba8a-ec68-405a-886b-1757b8a8c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:19.073176 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15ba8a-ec68-405a-886b-1757b8a8c3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.073438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15ba8a-ec68-405a-886b-1757b8a8c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.073713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15ba8a-ec68-405a-886b-1757b8a8c3a4 admin admin] PATH_INFO: `/v3/projects/ed66b40543004b4c921ac8dd3f395cc1/users/20fa6973da77400aad6483377028ccbf/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.087993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15ba8a-ec68-405a-886b-1757b8a8c3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed66b40543004b4c921ac8dd3f395cc1, user_id=20fa6973da77400aad6483377028ccbf, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:19.088961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WARNING py.warnings [None req-ad15ba8a-ec68-405a-886b-1757b8a8c3a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:21:19.088961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: warnings.warn(msg) May 13 13:21:19.088961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: May 13 13:21:19.090979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15ba8a-ec68-405a-886b-1757b8a8c3a4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:19.153899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 759/1501] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:21:19 2022] PUT /identity/v3/projects/ed66b40543004b4c921ac8dd3f395cc1/users/20fa6973da77400aad6483377028ccbf/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:21:19.165069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355b332-3efd-4562-9c71-8a478c53dcdd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.165360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355b332-3efd-4562-9c71-8a478c53dcdd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.165643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355b332-3efd-4562-9c71-8a478c53dcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.193344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3355b332-3efd-4562-9c71-8a478c53dcdd None None] MFA Rules not processed for user `20fa6973da77400aad6483377028ccbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:21:19.194729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3355b332-3efd-4562-9c71-8a478c53dcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:21:19.259412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 743/1502] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:21:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:21:19.272493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06a8f-c417-4eb2-b3f8-79ebf141d896 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.272822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06a8f-c417-4eb2-b3f8-79ebf141d896 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.273109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb06a8f-c417-4eb2-b3f8-79ebf141d896 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.274508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 760/1503] 10.222.0.25 () {54 vars in 963 bytes} [Fri May 13 13:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:21:19.279339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4052-542b-4839-be44-d9bcf548a6f7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.279665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4052-542b-4839-be44-d9bcf548a6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.279950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4052-542b-4839-be44-d9bcf548a6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.298425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-756c4052-542b-4839-be44-d9bcf548a6f7 None None] MFA Rules not processed for user `a7955ca85d92485d91abfa401b63f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:21:19.299608 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-756c4052-542b-4839-be44-d9bcf548a6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:21:19.348380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 744/1504] 10.222.0.25 () {58 vars in 1060 bytes} [Fri May 13 13:21:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:21:19.356287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb06a8f-c417-4eb2-b3f8-79ebf141d896 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:19.367208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb06a8f-c417-4eb2-b3f8-79ebf141d896 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:19.370802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcca1ad-596b-4c95-ae9c-2bf78807582d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:19.371434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcca1ad-596b-4c95-ae9c-2bf78807582d service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.371718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcca1ad-596b-4c95-ae9c-2bf78807582d service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.371997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcca1ad-596b-4c95-ae9c-2bf78807582d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.379937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcca1ad-596b-4c95-ae9c-2bf78807582d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:19.381413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcca1ad-596b-4c95-ae9c-2bf78807582d service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:19.389917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 761/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:21:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:21:19.436347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4d572-65e0-46cd-9933-ab41f1aeec86 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.436642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4d572-65e0-46cd-9933-ab41f1aeec86 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.436928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4d572-65e0-46cd-9933-ab41f1aeec86 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.438254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 745/1506] 10.222.0.25 () {52 vars in 829 bytes} [Fri May 13 13:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:21:19.441757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcca1ad-596b-4c95-ae9c-2bf78807582d service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:19.502349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcca1ad-596b-4c95-ae9c-2bf78807582d service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:19.507068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:19.507979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.508267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.508545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] PATH_INFO: `/v3/users/20fa6973da77400aad6483377028ccbf` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.509735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fa6973da77400aad6483377028ccbf)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:19.510791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WARNING py.warnings [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:21:19.510791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: warnings.warn(msg) May 13 13:21:19.510791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: May 13 13:21:19.512003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:19.753810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fa6973da77400aad6483377028ccbf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:19.754484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fa6973da77400aad6483377028ccbf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:19.822921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fa6973da77400aad6483377028ccbf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:19.834898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fa6973da77400aad6483377028ccbf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:19.835637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fa6973da77400aad6483377028ccbf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:19.858968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 762/1507] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:21:19 2022] DELETE /identity/v3/users/20fa6973da77400aad6483377028ccbf => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:21:19.864026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcb457-5892-430f-95f7-fffd2ec608cd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:19.865006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcb457-5892-430f-95f7-fffd2ec608cd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:19.865302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcb457-5892-430f-95f7-fffd2ec608cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:19.883200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0ffcb457-5892-430f-95f7-fffd2ec608cd None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:21:19.884409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0ffcb457-5892-430f-95f7-fffd2ec608cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:21:19.940339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 746/1508] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:21:19.950021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:19.951408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:21:20.005940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f8867-ca7f-48bf-bf41-b57ebcefd67f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:20.009374 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359a33f-d959-4a50-b079-c62f13ea343a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:20.010041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359a33f-d959-4a50-b079-c62f13ea343a service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:20.010335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359a33f-d959-4a50-b079-c62f13ea343a service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:20.010610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359a33f-d959-4a50-b079-c62f13ea343a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:20.019443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359a33f-d959-4a50-b079-c62f13ea343a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:20.020882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359a33f-d959-4a50-b079-c62f13ea343a service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:20.030069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 763/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:21:20.590440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcb457-5892-430f-95f7-fffd2ec608cd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:21:20.591893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0ffcb457-5892-430f-95f7-fffd2ec608cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:21:20.605927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcb457-5892-430f-95f7-fffd2ec608cd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:21:20.610683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:21:20.611657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:21:20.611945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:21:20.612222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] PATH_INFO: `/v3/projects/ed66b40543004b4c921ac8dd3f395cc1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:21:20.613496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed66b40543004b4c921ac8dd3f395cc1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:21:20.614551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:21:20.614551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: warnings.warn(msg) May 13 13:21:20.614551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: May 13 13:21:20.615965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:21:20.747661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed66b40543004b4c921ac8dd3f395cc1', 'request_id': 'req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:20.754091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed66b40543004b4c921ac8dd3f395cc1', 'request_id': 'req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:20.845979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed66b40543004b4c921ac8dd3f395cc1', 'request_id': 'req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:20.864542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] The token cache is being invalidate because project ed66b40543004b4c921ac8dd3f395cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:21:20.865091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:20.865599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a8401062-4d75-43fb-9fff-ec85c6fdb1a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:21:20.867182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 747/1510] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:21:20 2022] DELETE /identity/v3/projects/ed66b40543004b4c921ac8dd3f395cc1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:23:04.927889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425f768-3b28-46f2-87be-67935eae6da3 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:04.928459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425f768-3b28-46f2-87be-67935eae6da3 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:04.928459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425f768-3b28-46f2-87be-67935eae6da3 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:04.929708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:04.934532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5113042-406a-404d-8818-d20d021306f8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:04.935011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5113042-406a-404d-8818-d20d021306f8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:04.935118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5113042-406a-404d-8818-d20d021306f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:04.953541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e5113042-406a-404d-8818-d20d021306f8 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:04.954769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e5113042-406a-404d-8818-d20d021306f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:05.010189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:23:05.183755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425f768-3b28-46f2-87be-67935eae6da3 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:05.185380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9425f768-3b28-46f2-87be-67935eae6da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:05.259247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425f768-3b28-46f2-87be-67935eae6da3 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:05.264713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8303-793b-4986-92d4-c144b69a75ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:05.265362 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8303-793b-4986-92d4-c144b69a75ac service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:05.265647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8303-793b-4986-92d4-c144b69a75ac service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:05.265964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8303-793b-4986-92d4-c144b69a75ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:05.274034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f8303-793b-4986-92d4-c144b69a75ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:05.275669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f8303-793b-4986-92d4-c144b69a75ac service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:05.285325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 765/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:23:05.907540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc26d7-c6fe-41ac-acc6-d1d46bc77201 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:05.908591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc26d7-c6fe-41ac-acc6-d1d46bc77201 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:05.909017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc26d7-c6fe-41ac-acc6-d1d46bc77201 None None] PATH_INFO: `/v3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:05.910767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 749/1514] 10.222.0.70 () {54 vars in 925 bytes} [Fri May 13 13:23:05 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:23:05.916674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d46645-d2ce-4be6-b9d4-b5d5ca14fd50 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:05.917042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d46645-d2ce-4be6-b9d4-b5d5ca14fd50 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:05.917393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d46645-d2ce-4be6-b9d4-b5d5ca14fd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:05.938462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-41d46645-d2ce-4be6-b9d4-b5d5ca14fd50 None None] MFA Rules not processed for user `310c5d3ba81044ac8f3f192728cc3e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:05.939702 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-41d46645-d2ce-4be6-b9d4-b5d5ca14fd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:06.001119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 766/1515] 10.222.0.70 () {58 vars in 1015 bytes} [Fri May 13 13:23:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:23:06.183974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0f4c6-389c-44d1-9f4d-78708215418c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:06.184490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0f4c6-389c-44d1-9f4d-78708215418c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:06.184600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0f4c6-389c-44d1-9f4d-78708215418c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:06.186119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 750/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:06.191847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49b7f5-04e4-48ca-a964-1b91c86c0cf1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:06.192142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49b7f5-04e4-48ca-a964-1b91c86c0cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:06.192423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49b7f5-04e4-48ca-a964-1b91c86c0cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:06.210130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-3f49b7f5-04e4-48ca-a964-1b91c86c0cf1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:06.211333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3f49b7f5-04e4-48ca-a964-1b91c86c0cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:06.219328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 767/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:23:06.377310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0f4c6-389c-44d1-9f4d-78708215418c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:06.391855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0f4c6-389c-44d1-9f4d-78708215418c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:06.397328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4a250-ce2d-448a-b456-fde34247500b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:06.397993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4a250-ce2d-448a-b456-fde34247500b service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:06.398284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4a250-ce2d-448a-b456-fde34247500b service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:06.398562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4a250-ce2d-448a-b456-fde34247500b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:06.406711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4a250-ce2d-448a-b456-fde34247500b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:06.408346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4a250-ce2d-448a-b456-fde34247500b service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:06.417819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 751/1518] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:23:10.093513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d403-51be-4f91-9e94-ff902dcbbb7d None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:10.094124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d403-51be-4f91-9e94-ff902dcbbb7d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:10.094124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d403-51be-4f91-9e94-ff902dcbbb7d None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:10.095497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 768/1519] 10.222.0.25 () {54 vars in 960 bytes} [Fri May 13 13:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:10.100632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c7572-5148-4aa0-ab8f-0bf5ce625def None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:10.101054 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c7572-5148-4aa0-ab8f-0bf5ce625def None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:10.101194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c7572-5148-4aa0-ab8f-0bf5ce625def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:10.119607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-391c7572-5148-4aa0-ab8f-0bf5ce625def None None] MFA Rules not processed for user `310c5d3ba81044ac8f3f192728cc3e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:10.120812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-391c7572-5148-4aa0-ab8f-0bf5ce625def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:10.129980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 752/1520] 10.222.0.25 () {58 vars in 1057 bytes} [Fri May 13 13:23:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:23:10.138719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d403-51be-4f91-9e94-ff902dcbbb7d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:10.151814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d403-51be-4f91-9e94-ff902dcbbb7d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:10.156845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324fa80-6872-4310-ba0e-2fd1c3329390 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '310c5d3ba81044ac8f3f192728cc3e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:10.157496 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324fa80-6872-4310-ba0e-2fd1c3329390 service placement] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:10.157869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324fa80-6872-4310-ba0e-2fd1c3329390 service placement] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:10.158101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324fa80-6872-4310-ba0e-2fd1c3329390 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:10.166325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324fa80-6872-4310-ba0e-2fd1c3329390 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:10.167919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324fa80-6872-4310-ba0e-2fd1c3329390 service placement] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:10.177910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 769/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:23:17.767315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408aee-f5cc-4eb2-8127-787daf3805f0 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:17.767877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408aee-f5cc-4eb2-8127-787daf3805f0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:17.767877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a408aee-f5cc-4eb2-8127-787daf3805f0 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:17.769213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:17.774541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb318c8-28e2-42b8-8b2f-31fe58e1e7c1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:17.781562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb318c8-28e2-42b8-8b2f-31fe58e1e7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:17.781562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb318c8-28e2-42b8-8b2f-31fe58e1e7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:17.793943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-dcb318c8-28e2-42b8-8b2f-31fe58e1e7c1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:17.795142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-dcb318c8-28e2-42b8-8b2f-31fe58e1e7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:17.803476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:23:17.958934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408aee-f5cc-4eb2-8127-787daf3805f0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:17.973639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a408aee-f5cc-4eb2-8127-787daf3805f0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:17.978922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18277c-66ce-41a0-ba67-f55d428c9b9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:17.979548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18277c-66ce-41a0-ba67-f55d428c9b9b service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:17.979829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18277c-66ce-41a0-ba67-f55d428c9b9b service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:17.980105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18277c-66ce-41a0-ba67-f55d428c9b9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:17.988577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18277c-66ce-41a0-ba67-f55d428c9b9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:17.990200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18277c-66ce-41a0-ba67-f55d428c9b9b service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:17.999155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 754/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:23:18.904591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c680-8ab2-467b-8f4b-e5dd0c71b612 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:18.905288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c680-8ab2-467b-8f4b-e5dd0c71b612 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:18.905288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c680-8ab2-467b-8f4b-e5dd0c71b612 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:18.906536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 13 13:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:18.912076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0376a-fe4f-4a15-a538-d63c41bf982a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:18.912515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0376a-fe4f-4a15-a538-d63c41bf982a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:18.912672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0376a-fe4f-4a15-a538-d63c41bf982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:18.931870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9ba0376a-fe4f-4a15-a538-d63c41bf982a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:18.933062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9ba0376a-fe4f-4a15-a538-d63c41bf982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:18.940860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 13 13:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:23:19.089343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000c680-8ab2-467b-8f4b-e5dd0c71b612 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:19.105545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000c680-8ab2-467b-8f4b-e5dd0c71b612 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:19.111229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890203f1-b7a3-4251-a7da-6549cae1b6d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:19.111878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890203f1-b7a3-4251-a7da-6549cae1b6d8 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:19.112165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890203f1-b7a3-4251-a7da-6549cae1b6d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:19.112443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890203f1-b7a3-4251-a7da-6549cae1b6d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:19.120684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890203f1-b7a3-4251-a7da-6549cae1b6d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:19.122428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890203f1-b7a3-4251-a7da-6549cae1b6d8 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:19.131813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 772/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:23:23.317877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ec4f1-fa14-403e-b767-dfc9149f830b None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:23.318488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ec4f1-fa14-403e-b767-dfc9149f830b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:23.318488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ec4f1-fa14-403e-b767-dfc9149f830b None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:23.319813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:23.325264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1950f-fc75-44af-9a0b-74e6e8552cde None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:23.325667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1950f-fc75-44af-9a0b-74e6e8552cde None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:23.325851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1950f-fc75-44af-9a0b-74e6e8552cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:23.344394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d6f1950f-fc75-44af-9a0b-74e6e8552cde None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:23.346181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d6f1950f-fc75-44af-9a0b-74e6e8552cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:23.353722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:23:23.389651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ec4f1-fa14-403e-b767-dfc9149f830b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:23.391859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f97ec4f1-fa14-403e-b767-dfc9149f830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:23.448330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ec4f1-fa14-403e-b767-dfc9149f830b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:23.451811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013d6c0-f5f8-43a3-805e-b4113a4a5f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:23.452445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d6c0-f5f8-43a3-805e-b4113a4a5f7f service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:23.452724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d6c0-f5f8-43a3-805e-b4113a4a5f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:23.452995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d6c0-f5f8-43a3-805e-b4113a4a5f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:23.462074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013d6c0-f5f8-43a3-805e-b4113a4a5f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:23.463495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013d6c0-f5f8-43a3-805e-b4113a4a5f7f service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:23.473249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 757/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:23:25.643203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132b6de-8375-4ff2-bad1-fd1345aaf1ef None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:25.643751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132b6de-8375-4ff2-bad1-fd1345aaf1ef None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:25.643751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132b6de-8375-4ff2-bad1-fd1345aaf1ef None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:25.645176 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:25.650694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71983ef-baa1-4aff-8dc3-618a41f342da None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:25.651168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71983ef-baa1-4aff-8dc3-618a41f342da None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:25.651294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71983ef-baa1-4aff-8dc3-618a41f342da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:25.668612 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d71983ef-baa1-4aff-8dc3-618a41f342da None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:25.669819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d71983ef-baa1-4aff-8dc3-618a41f342da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:25.677953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:23:25.713123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132b6de-8375-4ff2-bad1-fd1345aaf1ef None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:25.723665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132b6de-8375-4ff2-bad1-fd1345aaf1ef None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:25.727286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b56fa8-46bc-4101-9c99-077eb35917bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:25.727975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56fa8-46bc-4101-9c99-077eb35917bd service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:25.728203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56fa8-46bc-4101-9c99-077eb35917bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:25.728478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b56fa8-46bc-4101-9c99-077eb35917bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:25.737663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b56fa8-46bc-4101-9c99-077eb35917bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:25.739114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b56fa8-46bc-4101-9c99-077eb35917bd service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:25.748383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 775/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:23:29.508921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76598cb-5464-44a3-b63f-11fedc5dd2e7 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:29.509460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76598cb-5464-44a3-b63f-11fedc5dd2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:29.509460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76598cb-5464-44a3-b63f-11fedc5dd2e7 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:29.510909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:29.516180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbdf3d-8933-4278-bbe3-55380e70bad0 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:29.516724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbdf3d-8933-4278-bbe3-55380e70bad0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:29.516724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbdf3d-8933-4278-bbe3-55380e70bad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:29.534527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-75cbdf3d-8933-4278-bbe3-55380e70bad0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:29.535844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-75cbdf3d-8933-4278-bbe3-55380e70bad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:29.543583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:23:29.823254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76598cb-5464-44a3-b63f-11fedc5dd2e7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:29.824927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d76598cb-5464-44a3-b63f-11fedc5dd2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:29.879804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76598cb-5464-44a3-b63f-11fedc5dd2e7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:29.883287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d05f89-0e0e-4691-9769-909eb5ff66a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:29.883926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05f89-0e0e-4691-9769-909eb5ff66a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:29.884208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05f89-0e0e-4691-9769-909eb5ff66a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:29.884484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d05f89-0e0e-4691-9769-909eb5ff66a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:29.893721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d05f89-0e0e-4691-9769-909eb5ff66a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:29.895166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d05f89-0e0e-4691-9769-909eb5ff66a8 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:29.904301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 760/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:23:32.240483 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07a22-7212-4ce0-8a64-d76023e2745c None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:32.241070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07a22-7212-4ce0-8a64-d76023e2745c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:32.241070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07a22-7212-4ce0-8a64-d76023e2745c None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:32.242455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 13 13:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:23:32.247607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d249f-e203-4294-a03d-da2589aee769 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:32.248084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d249f-e203-4294-a03d-da2589aee769 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:32.248194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d249f-e203-4294-a03d-da2589aee769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:32.266277 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5a3d249f-e203-4294-a03d-da2589aee769 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:23:32.267968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5a3d249f-e203-4294-a03d-da2589aee769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:32.275691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 13 13:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:23:32.548271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07a22-7212-4ce0-8a64-d76023e2745c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:23:32.550037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-82f07a22-7212-4ce0-8a64-d76023e2745c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:23:32.562817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07a22-7212-4ce0-8a64-d76023e2745c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:23:32.566934 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723daff-6497-43c1-826a-a8e27f4b463d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:23:32.567576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723daff-6497-43c1-826a-a8e27f4b463d service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:23:32.567863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723daff-6497-43c1-826a-a8e27f4b463d service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:23:32.568143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723daff-6497-43c1-826a-a8e27f4b463d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:23:32.576986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723daff-6497-43c1-826a-a8e27f4b463d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:23:32.578456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723daff-6497-43c1-826a-a8e27f4b463d service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:23:32.587599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 778/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:04.388061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88287b44-e3f9-4544-9797-2987fb6d25d5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.388802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88287b44-e3f9-4544-9797-2987fb6d25d5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.388802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88287b44-e3f9-4544-9797-2987fb6d25d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.423442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-88287b44-e3f9-4544-9797-2987fb6d25d5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.424684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-88287b44-e3f9-4544-9797-2987fb6d25d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.436920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 762/1540] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:04.447861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaf043-4c3f-4c5f-aea4-157b984590db None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.448463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaf043-4c3f-4c5f-aea4-157b984590db None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.448463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaf043-4c3f-4c5f-aea4-157b984590db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.478014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e5aaf043-4c3f-4c5f-aea4-157b984590db None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.479499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e5aaf043-4c3f-4c5f-aea4-157b984590db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.500243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 779/1541] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:04.505384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88287b44-e3f9-4544-9797-2987fb6d25d5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:04.521965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88287b44-e3f9-4544-9797-2987fb6d25d5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:04.534408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73cebb-08ae-4b86-b2c2-9ea9e338f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:04.535127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73cebb-08ae-4b86-b2c2-9ea9e338f0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.535417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73cebb-08ae-4b86-b2c2-9ea9e338f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.535702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73cebb-08ae-4b86-b2c2-9ea9e338f0a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.536473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73cebb-08ae-4b86-b2c2-9ea9e338f0a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:04.536785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73cebb-08ae-4b86-b2c2-9ea9e338f0a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:04.538486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73cebb-08ae-4b86-b2c2-9ea9e338f0a9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:04.556490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 763/1542] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:04.565508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaf043-4c3f-4c5f-aea4-157b984590db None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:04.576551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaf043-4c3f-4c5f-aea4-157b984590db None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:04.590133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b754db-2179-4f65-8905-1efb0bed7ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:04.590830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b754db-2179-4f65-8905-1efb0bed7ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.591812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b754db-2179-4f65-8905-1efb0bed7ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.592277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b754db-2179-4f65-8905-1efb0bed7ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.592984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b754db-2179-4f65-8905-1efb0bed7ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:04.595004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b754db-2179-4f65-8905-1efb0bed7ae9 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:04.600284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77548280-d9fd-4d07-b0cb-22bd7f7a95fa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.600906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77548280-d9fd-4d07-b0cb-22bd7f7a95fa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.601420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77548280-d9fd-4d07-b0cb-22bd7f7a95fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.627430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-77548280-d9fd-4d07-b0cb-22bd7f7a95fa None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.629020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-77548280-d9fd-4d07-b0cb-22bd7f7a95fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.646912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 764/1543] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:04.657481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15e860-ad55-4f98-a0ae-d266017d3cad None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.658153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15e860-ad55-4f98-a0ae-d266017d3cad None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.658659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15e860-ad55-4f98-a0ae-d266017d3cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.684452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3f15e860-ad55-4f98-a0ae-d266017d3cad None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.686049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3f15e860-ad55-4f98-a0ae-d266017d3cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.700376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 765/1544] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:04.702565 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f965c0-b0e9-4091-85bf-022d1eade8a1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.702859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f965c0-b0e9-4091-85bf-022d1eade8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.703149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f965c0-b0e9-4091-85bf-022d1eade8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.735563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-31f965c0-b0e9-4091-85bf-022d1eade8a1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.736815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-31f965c0-b0e9-4091-85bf-022d1eade8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.751572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 780/1545] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:04.751803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 766/1546] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:04.752758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f965c0-b0e9-4091-85bf-022d1eade8a1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:04.753238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365d738-cfe7-4cf8-bf01-c01f04da4853 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.753453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365d738-cfe7-4cf8-bf01-c01f04da4853 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.753740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365d738-cfe7-4cf8-bf01-c01f04da4853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.768717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f965c0-b0e9-4091-85bf-022d1eade8a1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:04.793064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-7365d738-cfe7-4cf8-bf01-c01f04da4853 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.794689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7365d738-cfe7-4cf8-bf01-c01f04da4853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.836088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:04.836996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.837288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.837571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.838593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:04.838914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:04.840544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:04.855736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 767/1547] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:04.856556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:04.866399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209930ee-c21f-4f59-8867-e0beafd2107e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:04.871197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c36a6-e251-4d56-999f-398ea062d93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:04.871866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c36a6-e251-4d56-999f-398ea062d93a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.872154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c36a6-e251-4d56-999f-398ea062d93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.872492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c36a6-e251-4d56-999f-398ea062d93a admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.873207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c36a6-e251-4d56-999f-398ea062d93a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:04.876305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c36a6-e251-4d56-999f-398ea062d93a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:04.881359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 781/1548] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:04.883070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8625-58b6-4988-89e5-5ce78d9463b9 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.883481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8625-58b6-4988-89e5-5ce78d9463b9 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.883870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8625-58b6-4988-89e5-5ce78d9463b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.902636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0c0c8625-58b6-4988-89e5-5ce78d9463b9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.904375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0c0c8625-58b6-4988-89e5-5ce78d9463b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.912278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 782/1549] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:04.913476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07b079-4329-4a23-92ac-a6acf4579079 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.913769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07b079-4329-4a23-92ac-a6acf4579079 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.914194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07b079-4329-4a23-92ac-a6acf4579079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.931274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bd07b079-4329-4a23-92ac-a6acf4579079 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.932419 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bd07b079-4329-4a23-92ac-a6acf4579079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.940104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 783/1550] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:04.941252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c84e1e-54ec-447f-af46-654f9984db54 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.941541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c84e1e-54ec-447f-af46-654f9984db54 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.941878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c84e1e-54ec-447f-af46-654f9984db54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.959477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-c8c84e1e-54ec-447f-af46-654f9984db54 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.959966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 768/1551] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:04.961827 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c8c84e1e-54ec-447f-af46-654f9984db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.961975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792e0d5-624c-4e64-a55e-e2134d65fd0e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.961975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792e0d5-624c-4e64-a55e-e2134d65fd0e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.962195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792e0d5-624c-4e64-a55e-e2134d65fd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.969804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 784/1552] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:04.971142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c84e1e-54ec-447f-af46-654f9984db54 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:04.980830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c84e1e-54ec-447f-af46-654f9984db54 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:04.982618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5792e0d5-624c-4e64-a55e-e2134d65fd0e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:04.983835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5792e0d5-624c-4e64-a55e-e2134d65fd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:04.985676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3dab6-6a3a-450e-b450-573dfd43c34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:04.986372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3dab6-6a3a-450e-b450-573dfd43c34e admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.986700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3dab6-6a3a-450e-b450-573dfd43c34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.986921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3dab6-6a3a-450e-b450-573dfd43c34e admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:04.987608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3dab6-6a3a-450e-b450-573dfd43c34e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:04.989527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3dab6-6a3a-450e-b450-573dfd43c34e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:04.992276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 769/1553] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:04.993370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d098-a71d-41c4-94da-9b1d7a4e6990 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:04.993658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d098-a71d-41c4-94da-9b1d7a4e6990 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:04.993976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d098-a71d-41c4-94da-9b1d7a4e6990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.012041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b565d098-a71d-41c4-94da-9b1d7a4e6990 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:05.013870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b565d098-a71d-41c4-94da-9b1d7a4e6990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:05.021923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 770/1554] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:05.023116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d1e7-a89c-4b57-b239-c84ef1157d43 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.023407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d1e7-a89c-4b57-b239-c84ef1157d43 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.023690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d1e7-a89c-4b57-b239-c84ef1157d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.041989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0535d1e7-a89c-4b57-b239-c84ef1157d43 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:05.043169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0535d1e7-a89c-4b57-b239-c84ef1157d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:05.051309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 771/1555] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:05.061503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d1e7-a89c-4b57-b239-c84ef1157d43 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.061503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d1e7-a89c-4b57-b239-c84ef1157d43 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.066204 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f8e7d-c545-41cd-9269-33b9790968c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.066911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f8e7d-c545-41cd-9269-33b9790968c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.067198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f8e7d-c545-41cd-9269-33b9790968c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.067478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f8e7d-c545-41cd-9269-33b9790968c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.068194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f8e7d-c545-41cd-9269-33b9790968c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:05.068499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f8e7d-c545-41cd-9269-33b9790968c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.070370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f8e7d-c545-41cd-9269-33b9790968c6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.082221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 772/1556] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:05.083393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48574196-a821-4ce3-98a3-149210729f4f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.083685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48574196-a821-4ce3-98a3-149210729f4f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.084047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48574196-a821-4ce3-98a3-149210729f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.097723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 785/1557] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:04 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:05.098604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3dab6-6a3a-450e-b450-573dfd43c34e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.102038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-48574196-a821-4ce3-98a3-149210729f4f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:05.103423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-48574196-a821-4ce3-98a3-149210729f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:05.161583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 773/1558] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:05.161827 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3dab6-6a3a-450e-b450-573dfd43c34e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.162813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd6af6-cae4-4122-ae29-c3c07516be07 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.163103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd6af6-cae4-4122-ae29-c3c07516be07 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.163516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd6af6-cae4-4122-ae29-c3c07516be07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.167333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e65c3-79ec-4eaf-9025-eaa8d7642685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.168001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e65c3-79ec-4eaf-9025-eaa8d7642685 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.168286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e65c3-79ec-4eaf-9025-eaa8d7642685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.168560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e65c3-79ec-4eaf-9025-eaa8d7642685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.169287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e65c3-79ec-4eaf-9025-eaa8d7642685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.171153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e65c3-79ec-4eaf-9025-eaa8d7642685 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.181088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 786/1559] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:05.187365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7cbd6af6-cae4-4122-ae29-c3c07516be07 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:05.187603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914679-7eb2-4b6b-9352-64ca74059e6f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.187705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7cbd6af6-cae4-4122-ae29-c3c07516be07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:05.187844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914679-7eb2-4b6b-9352-64ca74059e6f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.187844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914679-7eb2-4b6b-9352-64ca74059e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.191595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 774/1560] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:05.193049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813383b-15e4-449c-a51a-bfdc70e43f06 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.193342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813383b-15e4-449c-a51a-bfdc70e43f06 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.193627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813383b-15e4-449c-a51a-bfdc70e43f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.203979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-55914679-7eb2-4b6b-9352-64ca74059e6f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:05.205229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-55914679-7eb2-4b6b-9352-64ca74059e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:05.212169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a813383b-15e4-449c-a51a-bfdc70e43f06 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:05.213355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a813383b-15e4-449c-a51a-bfdc70e43f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:05.213486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 787/1561] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:05.214314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55914679-7eb2-4b6b-9352-64ca74059e6f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.223107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 775/1562] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:05.223747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813383b-15e4-449c-a51a-bfdc70e43f06 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.223858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55914679-7eb2-4b6b-9352-64ca74059e6f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.228456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.233047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.233047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.233047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.233047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:05.233047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.233538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WARNING py.warnings [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 13 13:25:05.233538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: warnings.warn(msg) May 13 13:25:05.233538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: May 13 13:25:05.233538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.233905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813383b-15e4-449c-a51a-bfdc70e43f06 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.238386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.239034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.239319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.239594 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.240418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.241421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:25:05.241421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: warnings.warn(msg) May 13 13:25:05.241421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: May 13 13:25:05.242626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.251725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 788/1563] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:05.251725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.255916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae0349-f234-487d-9560-da4416fef3b2 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.260527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.261317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.261605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.261923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.262606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.264182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WARNING py.warnings [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:25:05.264182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: warnings.warn(msg) May 13 13:25:05.264182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: May 13 13:25:05.265425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.345553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 776/1564] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/projects => generated 393 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:05.345553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.403328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c15a2c-d542-40c2-82b7-51be4deaf6ea admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.410262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.410804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.411090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.411367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.412154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:05.412408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.414124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.426305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 777/1565] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:05.427159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.438055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1f175-c697-41c2-a805-e69b88203247 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.443436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821e17-6674-42e5-a361-4ebda879368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.444384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821e17-6674-42e5-a361-4ebda879368a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.444820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821e17-6674-42e5-a361-4ebda879368a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.445504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821e17-6674-42e5-a361-4ebda879368a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.446585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821e17-6674-42e5-a361-4ebda879368a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.448318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821e17-6674-42e5-a361-4ebda879368a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.460074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 778/1566] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:05.461920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821e17-6674-42e5-a361-4ebda879368a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.471199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821e17-6674-42e5-a361-4ebda879368a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.476564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.477263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.477724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.478451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.478759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:05.479097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.480683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.493078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 779/1567] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:05.493898 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.504083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4171f-41f6-4988-9261-64c560880fc1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.508761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.509435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.509759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.513659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.513659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:05.513659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.513659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.525075 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 780/1568] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:05.525874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.535128 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b9e1-123f-484f-9570-1d08b3d01940 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.538460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 789/1569] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:05.538460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.542382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.542382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.542700 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.542985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.543870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:05.544017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.546157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.549122 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377f13a-3344-4fed-bd84-578ad9d99946 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.553955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17323828-329c-40e7-aa6f-82595a8451ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.554671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17323828-329c-40e7-aa6f-82595a8451ec admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.555063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17323828-329c-40e7-aa6f-82595a8451ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.555412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17323828-329c-40e7-aa6f-82595a8451ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.556172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17323828-329c-40e7-aa6f-82595a8451ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.558152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17323828-329c-40e7-aa6f-82595a8451ec admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.558292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 781/1570] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:05.558924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.569006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18035472-06a9-4a80-81f3-1a8f61abead3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.573966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b35a-ece2-4005-9a92-8da34bb9ec99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.582884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b35a-ece2-4005-9a92-8da34bb9ec99 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.582884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b35a-ece2-4005-9a92-8da34bb9ec99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.582884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b35a-ece2-4005-9a92-8da34bb9ec99 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.582884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2b35a-ece2-4005-9a92-8da34bb9ec99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.582884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2b35a-ece2-4005-9a92-8da34bb9ec99 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.658101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 790/1571] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:05.658919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17323828-329c-40e7-aa6f-82595a8451ec admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.668457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 782/1572] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:05.669955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b35a-ece2-4005-9a92-8da34bb9ec99 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.680296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b35a-ece2-4005-9a92-8da34bb9ec99 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.718799 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17323828-329c-40e7-aa6f-82595a8451ec admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.723422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ca047-501f-4454-8f78-5ac4841b3036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.724091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca047-501f-4454-8f78-5ac4841b3036 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.724377 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca047-501f-4454-8f78-5ac4841b3036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.724655 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca047-501f-4454-8f78-5ac4841b3036 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.725348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ca047-501f-4454-8f78-5ac4841b3036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.727301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ca047-501f-4454-8f78-5ac4841b3036 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.736634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.737540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.737853 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.738146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] PATH_INFO: `/v3/projects/942433b7b354400a95fdfe5614773495/users/b2cef0271718458190ef8c36325c1850/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.753518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=942433b7b354400a95fdfe5614773495, user_id=b2cef0271718458190ef8c36325c1850, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.755463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:25:05.755463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: warnings.warn(msg) May 13 13:25:05.755463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: May 13 13:25:05.756691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.826812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 791/1573] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/projects => generated 405 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:05.826812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ca047-501f-4454-8f78-5ac4841b3036 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.857423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 783/1574] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:05 2022] PUT /identity/v3/projects/942433b7b354400a95fdfe5614773495/users/b2cef0271718458190ef8c36325c1850/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:05.858611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:05.889008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ca047-501f-4454-8f78-5ac4841b3036 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.917699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75129d59-499d-4b1b-bca8-17d7ba4fb367 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:05.927385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fe95b-a7e3-4cb4-9858-abfea4a194b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.927385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fe95b-a7e3-4cb4-9858-abfea4a194b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.927385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fe95b-a7e3-4cb4-9858-abfea4a194b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.927385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fe95b-a7e3-4cb4-9858-abfea4a194b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.927996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fe95b-a7e3-4cb4-9858-abfea4a194b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.927996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fe95b-a7e3-4cb4-9858-abfea4a194b5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:05.943275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dfaa9-4fba-4b6d-b953-35c55889b6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:05.943971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dfaa9-4fba-4b6d-b953-35c55889b6ae admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:05.944258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dfaa9-4fba-4b6d-b953-35c55889b6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:05.944532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dfaa9-4fba-4b6d-b953-35c55889b6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:05.945494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dfaa9-4fba-4b6d-b953-35c55889b6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:05.947434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dfaa9-4fba-4b6d-b953-35c55889b6ae admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.029727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 784/1575] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:06.030596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fe95b-a7e3-4cb4-9858-abfea4a194b5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.063663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 792/1576] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:05 2022] POST /identity/v3/projects => generated 387 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:06.064521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dfaa9-4fba-4b6d-b953-35c55889b6ae admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.092455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fe95b-a7e3-4cb4-9858-abfea4a194b5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.126449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dfaa9-4fba-4b6d-b953-35c55889b6ae admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.130986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724dca7-7457-4ee3-b8e3-9ce25e737e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.138511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724dca7-7457-4ee3-b8e3-9ce25e737e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.138511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724dca7-7457-4ee3-b8e3-9ce25e737e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.138511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724dca7-7457-4ee3-b8e3-9ce25e737e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.138511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724dca7-7457-4ee3-b8e3-9ce25e737e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.138511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724dca7-7457-4ee3-b8e3-9ce25e737e73 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.148176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df949ee6-e33e-4e27-92dd-6ca7e6ce02cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.148845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949ee6-e33e-4e27-92dd-6ca7e6ce02cd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.149126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949ee6-e33e-4e27-92dd-6ca7e6ce02cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.149401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df949ee6-e33e-4e27-92dd-6ca7e6ce02cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.150191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df949ee6-e33e-4e27-92dd-6ca7e6ce02cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.151818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df949ee6-e33e-4e27-92dd-6ca7e6ce02cd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.162320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 785/1577] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.163081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df949ee6-e33e-4e27-92dd-6ca7e6ce02cd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.172762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df949ee6-e33e-4e27-92dd-6ca7e6ce02cd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.177230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5835c0-0093-4b44-b4e5-12fc5743b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.178775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5835c0-0093-4b44-b4e5-12fc5743b939 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.178775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5835c0-0093-4b44-b4e5-12fc5743b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.178775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5835c0-0093-4b44-b4e5-12fc5743b939 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.179109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5835c0-0093-4b44-b4e5-12fc5743b939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.180961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5835c0-0093-4b44-b4e5-12fc5743b939 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.254816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 786/1578] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:06 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:06.255684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5835c0-0093-4b44-b4e5-12fc5743b939 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.264673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5835c0-0093-4b44-b4e5-12fc5743b939 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.269106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91d60-b6dc-4473-b6f8-d74553e0e017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.269724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91d60-b6dc-4473-b6f8-d74553e0e017 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.270036 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91d60-b6dc-4473-b6f8-d74553e0e017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.270312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91d60-b6dc-4473-b6f8-d74553e0e017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.271018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e91d60-b6dc-4473-b6f8-d74553e0e017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.272567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e91d60-b6dc-4473-b6f8-d74553e0e017 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.282583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 787/1579] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.283332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91d60-b6dc-4473-b6f8-d74553e0e017 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.285624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 793/1580] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:06 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:06.287071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310af7d-e78c-493c-aab2-b8e88b7fe159 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.298060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310af7d-e78c-493c-aab2-b8e88b7fe159 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.298060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310af7d-e78c-493c-aab2-b8e88b7fe159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.298249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91d60-b6dc-4473-b6f8-d74553e0e017 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.314106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0310af7d-e78c-493c-aab2-b8e88b7fe159 None None] MFA Rules not processed for user `b2cef0271718458190ef8c36325c1850`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:06.315323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0310af7d-e78c-493c-aab2-b8e88b7fe159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:06.350872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb595f-a735-45c7-9447-daed3157fb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.351542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb595f-a735-45c7-9447-daed3157fb1c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.351821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb595f-a735-45c7-9447-daed3157fb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.352097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb595f-a735-45c7-9447-daed3157fb1c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.352819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb595f-a735-45c7-9447-daed3157fb1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.354775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb595f-a735-45c7-9447-daed3157fb1c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.381249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 794/1581] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:06.382092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310af7d-e78c-493c-aab2-b8e88b7fe159 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.391900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310af7d-e78c-493c-aab2-b8e88b7fe159 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.397066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc037a3f-2acc-490e-b3f5-abfe2248338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.397066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037a3f-2acc-490e-b3f5-abfe2248338f admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.397393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037a3f-2acc-490e-b3f5-abfe2248338f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.397645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc037a3f-2acc-490e-b3f5-abfe2248338f admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.398360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc037a3f-2acc-490e-b3f5-abfe2248338f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.400275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc037a3f-2acc-490e-b3f5-abfe2248338f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.456848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 788/1582] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:06 2022] POST /identity/v3/users => generated 303 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:06.457616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb595f-a735-45c7-9447-daed3157fb1c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.467905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb595f-a735-45c7-9447-daed3157fb1c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.489277 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a620f-bf45-4911-b696-a79b058b270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.494841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a620f-bf45-4911-b696-a79b058b270c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.494841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a620f-bf45-4911-b696-a79b058b270c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.494841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a620f-bf45-4911-b696-a79b058b270c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.494841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a620f-bf45-4911-b696-a79b058b270c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.494841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a620f-bf45-4911-b696-a79b058b270c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.499235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 795/1583] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:06 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:06.500289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc037a3f-2acc-490e-b3f5-abfe2248338f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.516826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc037a3f-2acc-490e-b3f5-abfe2248338f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.521750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefffefb-056a-4590-b08c-df085d918b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.529291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefffefb-056a-4590-b08c-df085d918b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.529291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefffefb-056a-4590-b08c-df085d918b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.529291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefffefb-056a-4590-b08c-df085d918b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.529291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefffefb-056a-4590-b08c-df085d918b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.529291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefffefb-056a-4590-b08c-df085d918b28 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.535881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 796/1584] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.536689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefffefb-056a-4590-b08c-df085d918b28 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.546796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefffefb-056a-4590-b08c-df085d918b28 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.551396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dffc16-e40c-4947-8626-5c1636e153dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.552073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dffc16-e40c-4947-8626-5c1636e153dc admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.552342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dffc16-e40c-4947-8626-5c1636e153dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.552619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dffc16-e40c-4947-8626-5c1636e153dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.553334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dffc16-e40c-4947-8626-5c1636e153dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.554975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dffc16-e40c-4947-8626-5c1636e153dc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.565840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 797/1585] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.566940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dffc16-e40c-4947-8626-5c1636e153dc admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.577004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dffc16-e40c-4947-8626-5c1636e153dc admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.581956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72071615-352c-4c34-b38e-73f8e0489292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.582608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72071615-352c-4c34-b38e-73f8e0489292 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.582926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72071615-352c-4c34-b38e-73f8e0489292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.583208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72071615-352c-4c34-b38e-73f8e0489292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.583935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72071615-352c-4c34-b38e-73f8e0489292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.585593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72071615-352c-4c34-b38e-73f8e0489292 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.594954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 789/1586] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:06 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:06.595957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a620f-bf45-4911-b696-a79b058b270c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.599672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 798/1587] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.599672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09b809-f73b-479c-82b5-01146f91b11a None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.599672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09b809-f73b-479c-82b5-01146f91b11a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.599672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09b809-f73b-479c-82b5-01146f91b11a None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.600461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 799/1588] 10.222.0.25 () {54 vars in 961 bytes} [Fri May 13 13:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:25:06.601224 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09b809-f73b-479c-82b5-01146f91b11a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.613699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a620f-bf45-4911-b696-a79b058b270c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.614735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09b809-f73b-479c-82b5-01146f91b11a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.619341 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8a37f-2055-47e5-aca7-c45e23bd27af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.619508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c803830-1940-4797-9bc7-99abdb45cf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.620189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c803830-1940-4797-9bc7-99abdb45cf28 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.620418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8a37f-2055-47e5-aca7-c45e23bd27af admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.620551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c803830-1940-4797-9bc7-99abdb45cf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.620660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8a37f-2055-47e5-aca7-c45e23bd27af admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.620763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c803830-1940-4797-9bc7-99abdb45cf28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.620891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8a37f-2055-47e5-aca7-c45e23bd27af admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.621453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c803830-1940-4797-9bc7-99abdb45cf28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.621577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8a37f-2055-47e5-aca7-c45e23bd27af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.623120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c803830-1940-4797-9bc7-99abdb45cf28 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.623794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8a37f-2055-47e5-aca7-c45e23bd27af admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.634382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 800/1589] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.638087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c803830-1940-4797-9bc7-99abdb45cf28 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.647855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c803830-1940-4797-9bc7-99abdb45cf28 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.652718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad9f3a-3a81-4505-8ac2-c11be1878433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.653380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9f3a-3a81-4505-8ac2-c11be1878433 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.654218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9f3a-3a81-4505-8ac2-c11be1878433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.654498 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9f3a-3a81-4505-8ac2-c11be1878433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.655220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad9f3a-3a81-4505-8ac2-c11be1878433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.656852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad9f3a-3a81-4505-8ac2-c11be1878433 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.670564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 801/1590] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.670564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad9f3a-3a81-4505-8ac2-c11be1878433 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.679032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad9f3a-3a81-4505-8ac2-c11be1878433 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.684011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82911cdf-aa0d-4e1c-9341-71dab1caebc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.684961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911cdf-aa0d-4e1c-9341-71dab1caebc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.685320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911cdf-aa0d-4e1c-9341-71dab1caebc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.685669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911cdf-aa0d-4e1c-9341-71dab1caebc2 admin admin] PATH_INFO: `/v3/projects/4cd4aa315c4340a1a649d8c51cea0da9/users/08b9fbf87cfa417fbea857ae602a140e/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.701558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82911cdf-aa0d-4e1c-9341-71dab1caebc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd4aa315c4340a1a649d8c51cea0da9, user_id=08b9fbf87cfa417fbea857ae602a140e, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.704275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82911cdf-aa0d-4e1c-9341-71dab1caebc2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.807560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 790/1591] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:06 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:06.807560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8a37f-2055-47e5-aca7-c45e23bd27af admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.815801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8a37f-2055-47e5-aca7-c45e23bd27af admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.821310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974d757-d9ed-4b60-a479-a8376510eef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.822138 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974d757-d9ed-4b60-a479-a8376510eef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.825915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974d757-d9ed-4b60-a479-a8376510eef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.825915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974d757-d9ed-4b60-a479-a8376510eef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.825915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974d757-d9ed-4b60-a479-a8376510eef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.826370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974d757-d9ed-4b60-a479-a8376510eef1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.838798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 791/1592] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.839637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974d757-d9ed-4b60-a479-a8376510eef1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.845665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 802/1593] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:06 2022] PUT /identity/v3/projects/4cd4aa315c4340a1a649d8c51cea0da9/users/08b9fbf87cfa417fbea857ae602a140e/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:06.846713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82911cdf-aa0d-4e1c-9341-71dab1caebc2 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:06.850127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974d757-d9ed-4b60-a479-a8376510eef1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.906058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a6413-715b-4b51-b23f-276a7ca24f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.906509 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82911cdf-aa0d-4e1c-9341-71dab1caebc2 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:06.906735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a6413-715b-4b51-b23f-276a7ca24f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.908037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a6413-715b-4b51-b23f-276a7ca24f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.908037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a6413-715b-4b51-b23f-276a7ca24f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.908037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a6413-715b-4b51-b23f-276a7ca24f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.909677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a6413-715b-4b51-b23f-276a7ca24f4b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.912119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098409f8-5fdc-4ff3-b23f-4e0802e8d3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:06.913011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098409f8-5fdc-4ff3-b23f-4e0802e8d3b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.913294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098409f8-5fdc-4ff3-b23f-4e0802e8d3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.913570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098409f8-5fdc-4ff3-b23f-4e0802e8d3b4 admin admin] PATH_INFO: `/v3/projects/e25902914d244bfeaa038f24838c28ca/users/f56c908ada31445e85ec1dfee8e2ebdb/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.919715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 792/1594] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:06.920952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788866b-72c7-46a2-a3ad-f732b541e9ce None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:06.921652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788866b-72c7-46a2-a3ad-f732b541e9ce None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:06.921652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788866b-72c7-46a2-a3ad-f732b541e9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:06.928259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098409f8-5fdc-4ff3-b23f-4e0802e8d3b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e25902914d244bfeaa038f24838c28ca, user_id=f56c908ada31445e85ec1dfee8e2ebdb, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:06.930979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098409f8-5fdc-4ff3-b23f-4e0802e8d3b4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:06.938910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c788866b-72c7-46a2-a3ad-f732b541e9ce None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:06.940122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c788866b-72c7-46a2-a3ad-f732b541e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:07.013890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 793/1595] 10.222.0.25 () {58 vars in 1058 bytes} [Fri May 13 13:25:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:07.014820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788866b-72c7-46a2-a3ad-f732b541e9ce None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.024389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788866b-72c7-46a2-a3ad-f732b541e9ce None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.029611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8027f5c-1b2a-4ed1-9c9f-9803612273a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.030289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027f5c-1b2a-4ed1-9c9f-9803612273a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.030599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027f5c-1b2a-4ed1-9c9f-9803612273a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.030879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027f5c-1b2a-4ed1-9c9f-9803612273a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.031597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8027f5c-1b2a-4ed1-9c9f-9803612273a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.033212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8027f5c-1b2a-4ed1-9c9f-9803612273a4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.043016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 794/1596] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:07.043873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8027f5c-1b2a-4ed1-9c9f-9803612273a4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.053846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8027f5c-1b2a-4ed1-9c9f-9803612273a4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.055536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 803/1597] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:06 2022] PUT /identity/v3/projects/e25902914d244bfeaa038f24838c28ca/users/f56c908ada31445e85ec1dfee8e2ebdb/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:07.059320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098409f8-5fdc-4ff3-b23f-4e0802e8d3b4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.115301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25e6b7-d393-4bc9-b926-cabf5c9650d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.116010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25e6b7-d393-4bc9-b926-cabf5c9650d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.116299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25e6b7-d393-4bc9-b926-cabf5c9650d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.116583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25e6b7-d393-4bc9-b926-cabf5c9650d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.117324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25e6b7-d393-4bc9-b926-cabf5c9650d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.118977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25e6b7-d393-4bc9-b926-cabf5c9650d9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.124828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098409f8-5fdc-4ff3-b23f-4e0802e8d3b4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.130049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 795/1598] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:07.131166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25e6b7-d393-4bc9-b926-cabf5c9650d9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.134280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96478f-b8e4-4a48-868a-88fcea920eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.135038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96478f-b8e4-4a48-868a-88fcea920eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.135565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96478f-b8e4-4a48-868a-88fcea920eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.135935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96478f-b8e4-4a48-868a-88fcea920eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.136741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96478f-b8e4-4a48-868a-88fcea920eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.138972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96478f-b8e4-4a48-868a-88fcea920eb2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.145223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25e6b7-d393-4bc9-b926-cabf5c9650d9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.152708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 804/1599] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:07.152871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf1d24-313e-4297-8a81-6df3112402c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.152871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf1d24-313e-4297-8a81-6df3112402c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.160415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30ad8a-730f-4994-9692-df19c43ff522 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.160526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf1d24-313e-4297-8a81-6df3112402c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.160635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30ad8a-730f-4994-9692-df19c43ff522 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.160727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf1d24-313e-4297-8a81-6df3112402c7 admin admin] PATH_INFO: `/v3/projects/e31f5d197d024572b044da9343a9b4a1/users/64db0dfebf94402d978d9b0d0639a695/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.160817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30ad8a-730f-4994-9692-df19c43ff522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.169673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf1d24-313e-4297-8a81-6df3112402c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31f5d197d024572b044da9343a9b4a1, user_id=64db0dfebf94402d978d9b0d0639a695, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.170184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6d30ad8a-730f-4994-9692-df19c43ff522 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:07.171796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6d30ad8a-730f-4994-9692-df19c43ff522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:07.172784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf1d24-313e-4297-8a81-6df3112402c7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.180285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 805/1600] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:07.181228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30ad8a-730f-4994-9692-df19c43ff522 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.190878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30ad8a-730f-4994-9692-df19c43ff522 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.195929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d66c-a461-4889-8ae6-0b56a34780fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.196591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d66c-a461-4889-8ae6-0b56a34780fe admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.196878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d66c-a461-4889-8ae6-0b56a34780fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.197156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d66c-a461-4889-8ae6-0b56a34780fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.198002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d66c-a461-4889-8ae6-0b56a34780fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.199643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d66c-a461-4889-8ae6-0b56a34780fe admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.210217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 806/1601] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:07.211751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d66c-a461-4889-8ae6-0b56a34780fe admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.251436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 796/1602] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:07 2022] PUT /identity/v3/projects/e31f5d197d024572b044da9343a9b4a1/users/64db0dfebf94402d978d9b0d0639a695/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:07.252430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf1d24-313e-4297-8a81-6df3112402c7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.279305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d66c-a461-4889-8ae6-0b56a34780fe admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.312567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf1d24-313e-4297-8a81-6df3112402c7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.317206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9514e-3dec-47d0-a351-799692746240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.318123 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9514e-3dec-47d0-a351-799692746240 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.318439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9514e-3dec-47d0-a351-799692746240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.318717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9514e-3dec-47d0-a351-799692746240 admin admin] PATH_INFO: `/v3/projects/27638a23724b420680f3287c88580d31/users/3755f970272c4d528019826cc7bc73f1/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.333408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9514e-3dec-47d0-a351-799692746240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27638a23724b420680f3287c88580d31, user_id=3755f970272c4d528019826cc7bc73f1, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.336110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9514e-3dec-47d0-a351-799692746240 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.340813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90114dbf-5247-4f5c-9fdf-28b18de4c007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.341475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90114dbf-5247-4f5c-9fdf-28b18de4c007 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.341762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90114dbf-5247-4f5c-9fdf-28b18de4c007 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.342063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90114dbf-5247-4f5c-9fdf-28b18de4c007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.395000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90114dbf-5247-4f5c-9fdf-28b18de4c007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.396594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90114dbf-5247-4f5c-9fdf-28b18de4c007 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.405347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 807/1603] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:07.406714 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920554d2-0c7f-47cd-ad04-0ef2d8d5fa54 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.407003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920554d2-0c7f-47cd-ad04-0ef2d8d5fa54 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.407280 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920554d2-0c7f-47cd-ad04-0ef2d8d5fa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.417248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 797/1604] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:07 2022] PUT /identity/v3/projects/27638a23724b420680f3287c88580d31/users/3755f970272c4d528019826cc7bc73f1/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:07.418369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9514e-3dec-47d0-a351-799692746240 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.426796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-920554d2-0c7f-47cd-ad04-0ef2d8d5fa54 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:07.427986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-920554d2-0c7f-47cd-ad04-0ef2d8d5fa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:07.486086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9514e-3dec-47d0-a351-799692746240 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.491236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6fce7-0eed-4a23-a60a-684841cc528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.491408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 808/1605] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:07.492232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920554d2-0c7f-47cd-ad04-0ef2d8d5fa54 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.494345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6fce7-0eed-4a23-a60a-684841cc528e admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.495014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6fce7-0eed-4a23-a60a-684841cc528e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.495510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6fce7-0eed-4a23-a60a-684841cc528e admin admin] PATH_INFO: `/v3/projects/915a7ff1f6d64480a08c38e762da4134/users/233f5342cf334c7689cb9e71f4fbaab3/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.502791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920554d2-0c7f-47cd-ad04-0ef2d8d5fa54 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.508035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697780a-3bef-42e9-acf1-8f9facce279d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.508702 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697780a-3bef-42e9-acf1-8f9facce279d admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.508988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697780a-3bef-42e9-acf1-8f9facce279d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.509265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697780a-3bef-42e9-acf1-8f9facce279d admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.510021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697780a-3bef-42e9-acf1-8f9facce279d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.511669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697780a-3bef-42e9-acf1-8f9facce279d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.516904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6fce7-0eed-4a23-a60a-684841cc528e admin admin] RBAC: Authorizing `identity:create_grant(project_id=915a7ff1f6d64480a08c38e762da4134, user_id=233f5342cf334c7689cb9e71f4fbaab3, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.520091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6fce7-0eed-4a23-a60a-684841cc528e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.522699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 809/1606] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:07.523629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697780a-3bef-42e9-acf1-8f9facce279d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.575064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697780a-3bef-42e9-acf1-8f9facce279d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.579333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f62213-a584-418c-b4df-f87463993690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.579971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f62213-a584-418c-b4df-f87463993690 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.580255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f62213-a584-418c-b4df-f87463993690 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.580531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f62213-a584-418c-b4df-f87463993690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.589268 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f62213-a584-418c-b4df-f87463993690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.590750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f62213-a584-418c-b4df-f87463993690 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.599979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 810/1607] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:07.600806 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f62213-a584-418c-b4df-f87463993690 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.610183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f62213-a584-418c-b4df-f87463993690 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.615222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6d18-0d41-4542-91f6-c60bd5a5e99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.616128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6d18-0d41-4542-91f6-c60bd5a5e99e admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.616410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6d18-0d41-4542-91f6-c60bd5a5e99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.616688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6d18-0d41-4542-91f6-c60bd5a5e99e admin admin] PATH_INFO: `/v3/projects/b6d5e50de93044abbe8157e2085f63a6/users/dddb85c15f2d4606921b0818a296b296/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.628413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 798/1608] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:07 2022] PUT /identity/v3/projects/915a7ff1f6d64480a08c38e762da4134/users/233f5342cf334c7689cb9e71f4fbaab3/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:07.629815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21746a9-2a09-4010-8e76-66adad048036 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.630127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21746a9-2a09-4010-8e76-66adad048036 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.630401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21746a9-2a09-4010-8e76-66adad048036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.640800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c6d18-0d41-4542-91f6-c60bd5a5e99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d5e50de93044abbe8157e2085f63a6, user_id=dddb85c15f2d4606921b0818a296b296, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.640800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c6d18-0d41-4542-91f6-c60bd5a5e99e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.667179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e21746a9-2a09-4010-8e76-66adad048036 None None] MFA Rules not processed for user `64db0dfebf94402d978d9b0d0639a695`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:07.668378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e21746a9-2a09-4010-8e76-66adad048036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:07.730355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 799/1609] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:07.733989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055f0a5-13ed-4e51-b4a1-65249c97eaa4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.733989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055f0a5-13ed-4e51-b4a1-65249c97eaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.733989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055f0a5-13ed-4e51-b4a1-65249c97eaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.751063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5055f0a5-13ed-4e51-b4a1-65249c97eaa4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:07.752597 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5055f0a5-13ed-4e51-b4a1-65249c97eaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:07.782502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 811/1610] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:07 2022] PUT /identity/v3/projects/b6d5e50de93044abbe8157e2085f63a6/users/dddb85c15f2d4606921b0818a296b296/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:07.783845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6d18-0d41-4542-91f6-c60bd5a5e99e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.815068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 800/1611] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:07.828002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055f0a5-13ed-4e51-b4a1-65249c97eaa4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:07.840118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055f0a5-13ed-4e51-b4a1-65249c97eaa4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.857699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6d18-0d41-4542-91f6-c60bd5a5e99e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:07.863406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6203c-db84-4f57-bd3a-9d2851090161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.864268 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6203c-db84-4f57-bd3a-9d2851090161 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.864652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6203c-db84-4f57-bd3a-9d2851090161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.865067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6203c-db84-4f57-bd3a-9d2851090161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.918717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040a57-fc51-4aca-b43a-75820b75984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:07.920159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040a57-fc51-4aca-b43a-75820b75984a admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.920439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040a57-fc51-4aca-b43a-75820b75984a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.920802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040a57-fc51-4aca-b43a-75820b75984a admin admin] PATH_INFO: `/v3/projects/65563e9192834a848521c69c7c80f078/users/80623f3527844489a5e31a786080fa10/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.938152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09040a57-fc51-4aca-b43a-75820b75984a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65563e9192834a848521c69c7c80f078, user_id=80623f3527844489a5e31a786080fa10, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.940705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09040a57-fc51-4aca-b43a-75820b75984a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.949643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6203c-db84-4f57-bd3a-9d2851090161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:07.951399 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6203c-db84-4f57-bd3a-9d2851090161 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:07.962003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 812/1612] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:07.979269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63973a7d-391c-4941-8d92-bba01d09c0f2 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:07.979269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63973a7d-391c-4941-8d92-bba01d09c0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:07.979269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63973a7d-391c-4941-8d92-bba01d09c0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:07.983153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-63973a7d-391c-4941-8d92-bba01d09c0f2 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:07.984370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-63973a7d-391c-4941-8d92-bba01d09c0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:07.992731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 813/1613] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:07.993522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63973a7d-391c-4941-8d92-bba01d09c0f2 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.006856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63973a7d-391c-4941-8d92-bba01d09c0f2 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.011782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.012446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.012735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.013379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.014534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.016864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.057498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 801/1614] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:07 2022] PUT /identity/v3/projects/65563e9192834a848521c69c7c80f078/users/80623f3527844489a5e31a786080fa10/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:08.058871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823063cf-9531-499f-bb6a-61dacb3041df None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.059133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823063cf-9531-499f-bb6a-61dacb3041df None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.059420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823063cf-9531-499f-bb6a-61dacb3041df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.087021 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-823063cf-9531-499f-bb6a-61dacb3041df None None] MFA Rules not processed for user `dddb85c15f2d4606921b0818a296b296`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:08.088222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-823063cf-9531-499f-bb6a-61dacb3041df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:08.103909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 814/1615] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:07 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:08.105327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.106678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:08.166374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 802/1616] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:08.169071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a02967-ea99-45d2-8794-7a58bb31b78e None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.169071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a02967-ea99-45d2-8794-7a58bb31b78e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.169071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a02967-ea99-45d2-8794-7a58bb31b78e None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.174316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baf5ac-a628-4361-b2e4-20edda24137d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.174854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 803/1617] 10.222.0.25 () {54 vars in 961 bytes} [Fri May 13 13:25:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:25:08.175820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a02967-ea99-45d2-8794-7a58bb31b78e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.178835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60b995-52d0-4f54-8de1-174856f9f401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.179691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b995-52d0-4f54-8de1-174856f9f401 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.180053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b995-52d0-4f54-8de1-174856f9f401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.180442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60b995-52d0-4f54-8de1-174856f9f401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.187350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a02967-ea99-45d2-8794-7a58bb31b78e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.191635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9b6e0-8d2a-4b96-824a-9011dd9915db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.197769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b6e0-8d2a-4b96-824a-9011dd9915db service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.197769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b6e0-8d2a-4b96-824a-9011dd9915db service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.197769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b6e0-8d2a-4b96-824a-9011dd9915db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.248787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60b995-52d0-4f54-8de1-174856f9f401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.250389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60b995-52d0-4f54-8de1-174856f9f401 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.258284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9b6e0-8d2a-4b96-824a-9011dd9915db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.260392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9b6e0-8d2a-4b96-824a-9011dd9915db service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.266217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 815/1618] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:08.267720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8ebe2-6257-4618-9934-99d3689290cb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.268017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8ebe2-6257-4618-9934-99d3689290cb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.268288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8ebe2-6257-4618-9934-99d3689290cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.277634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 804/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:08.279374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9b6e0-8d2a-4b96-824a-9011dd9915db service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.287971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-96e8ebe2-6257-4618-9934-99d3689290cb None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:08.289393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-96e8ebe2-6257-4618-9934-99d3689290cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:08.293775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9b6e0-8d2a-4b96-824a-9011dd9915db service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.300474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 816/1620] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:08.301629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8ebe2-6257-4618-9934-99d3689290cb None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.302192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a50f2-521f-4dab-bf85-af45a31815ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.303131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a50f2-521f-4dab-bf85-af45a31815ff admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.303967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a50f2-521f-4dab-bf85-af45a31815ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.304448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a50f2-521f-4dab-bf85-af45a31815ff admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.305604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a50f2-521f-4dab-bf85-af45a31815ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.308215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a50f2-521f-4dab-bf85-af45a31815ff admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.315092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8ebe2-6257-4618-9934-99d3689290cb None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.324000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4184e5-db25-427b-813f-077839e2bd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.324772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4184e5-db25-427b-813f-077839e2bd62 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.325297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4184e5-db25-427b-813f-077839e2bd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.326228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4184e5-db25-427b-813f-077839e2bd62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.330063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4184e5-db25-427b-813f-077839e2bd62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.330878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4184e5-db25-427b-813f-077839e2bd62 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.399065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 805/1621] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:08 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:08.399065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b149d27-2420-4b91-8490-207955115122 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.399065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b149d27-2420-4b91-8490-207955115122 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.399065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b149d27-2420-4b91-8490-207955115122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.415659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7b149d27-2420-4b91-8490-207955115122 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:08.416927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7b149d27-2420-4b91-8490-207955115122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:08.432276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 817/1622] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:08 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:08.432276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4184e5-db25-427b-813f-077839e2bd62 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.521244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 806/1623] 10.222.0.25 () {58 vars in 1058 bytes} [Fri May 13 13:25:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:08.522813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b149d27-2420-4b91-8490-207955115122 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.539047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4184e5-db25-427b-813f-077839e2bd62 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.542146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b149d27-2420-4b91-8490-207955115122 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.544923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badcc4b-fb6f-43c8-b0a0-9da0361e2b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.546171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badcc4b-fb6f-43c8-b0a0-9da0361e2b48 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.546384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badcc4b-fb6f-43c8-b0a0-9da0361e2b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.546747 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badcc4b-fb6f-43c8-b0a0-9da0361e2b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.625550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badcc4b-fb6f-43c8-b0a0-9da0361e2b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.628069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badcc4b-fb6f-43c8-b0a0-9da0361e2b48 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.644304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 818/1624] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:08.645357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badcc4b-fb6f-43c8-b0a0-9da0361e2b48 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.656814 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27057a-e8f1-485d-a292-7cf087bd2291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.657282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badcc4b-fb6f-43c8-b0a0-9da0361e2b48 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.657508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27057a-e8f1-485d-a292-7cf087bd2291 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.657834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27057a-e8f1-485d-a292-7cf087bd2291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.658113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27057a-e8f1-485d-a292-7cf087bd2291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.659534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27057a-e8f1-485d-a292-7cf087bd2291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.662043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27057a-e8f1-485d-a292-7cf087bd2291 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.665488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a8788-28cb-4ef4-96b3-b61e7d1b5b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.666255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a8788-28cb-4ef4-96b3-b61e7d1b5b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.668594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a8788-28cb-4ef4-96b3-b61e7d1b5b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.668594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a8788-28cb-4ef4-96b3-b61e7d1b5b23 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.668594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a8788-28cb-4ef4-96b3-b61e7d1b5b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.670628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a8788-28cb-4ef4-96b3-b61e7d1b5b23 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.677603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 807/1625] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:08.681452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa3134-34d6-443e-bd79-544e4dd92eda None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.682103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa3134-34d6-443e-bd79-544e4dd92eda None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.682875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa3134-34d6-443e-bd79-544e4dd92eda None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.684658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 808/1626] 10.222.0.25 () {54 vars in 912 bytes} [Fri May 13 13:25:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:25:08.686352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa3134-34d6-443e-bd79-544e4dd92eda None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.703185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa3134-34d6-443e-bd79-544e4dd92eda None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.708566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d3235-50c4-4845-8f03-163dd062e191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.709623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d3235-50c4-4845-8f03-163dd062e191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.709962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d3235-50c4-4845-8f03-163dd062e191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.710292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d3235-50c4-4845-8f03-163dd062e191 admin admin] PATH_INFO: `/v3/projects/03c2fc33ceb84ef5b6a4c5b6027c91b4/users/181bf5b37b81456e908f699c07240175/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.728674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d3235-50c4-4845-8f03-163dd062e191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c2fc33ceb84ef5b6a4c5b6027c91b4, user_id=181bf5b37b81456e908f699c07240175, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:08.731535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d3235-50c4-4845-8f03-163dd062e191 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:08.757961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 819/1627] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:08 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:08.759087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7ff19-54ac-4780-b07e-82251711af83 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.759382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7ff19-54ac-4780-b07e-82251711af83 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.759662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7ff19-54ac-4780-b07e-82251711af83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.779160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-49f7ff19-54ac-4780-b07e-82251711af83 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:08.780396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-49f7ff19-54ac-4780-b07e-82251711af83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:08.861095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 820/1628] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:25:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:08.862182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7ff19-54ac-4780-b07e-82251711af83 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.876689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 809/1629] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:08 2022] PUT /identity/v3/projects/03c2fc33ceb84ef5b6a4c5b6027c91b4/users/181bf5b37b81456e908f699c07240175/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:08.877915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d3235-50c4-4845-8f03-163dd062e191 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:08.886788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7ff19-54ac-4780-b07e-82251711af83 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.960219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d3235-50c4-4845-8f03-163dd062e191 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:08.964335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fb334-24f2-4a55-b102-a6c16d77702f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.967160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb334-24f2-4a55-b102-a6c16d77702f service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.967160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb334-24f2-4a55-b102-a6c16d77702f service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.967160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb334-24f2-4a55-b102-a6c16d77702f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:08.997890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae965-1542-4761-aef2-b78879c31f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:08.998974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae965-1542-4761-aef2-b78879c31f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:08.999260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae965-1542-4761-aef2-b78879c31f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:08.999538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ae965-1542-4761-aef2-b78879c31f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.000558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ae965-1542-4761-aef2-b78879c31f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.002424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ae965-1542-4761-aef2-b78879c31f24 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.018686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 821/1630] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:09.018686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae965-1542-4761-aef2-b78879c31f24 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.032486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ae965-1542-4761-aef2-b78879c31f24 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:09.039090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5620a16-9c01-40dd-a319-4382b3a62d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:09.039767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620a16-9c01-40dd-a319-4382b3a62d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.041813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620a16-9c01-40dd-a319-4382b3a62d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.041813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620a16-9c01-40dd-a319-4382b3a62d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.041813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5620a16-9c01-40dd-a319-4382b3a62d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.043070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5620a16-9c01-40dd-a319-4382b3a62d33 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.058334 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fb334-24f2-4a55-b102-a6c16d77702f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.062079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fb334-24f2-4a55-b102-a6c16d77702f service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.069835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 822/1631] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:09.076292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5620a16-9c01-40dd-a319-4382b3a62d33 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.080879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 810/1632] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:09.082431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fb334-24f2-4a55-b102-a6c16d77702f service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.091691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5620a16-9c01-40dd-a319-4382b3a62d33 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:09.095190 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fb334-24f2-4a55-b102-a6c16d77702f service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:09.101070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41617315-c423-4d16-8e33-a15444797b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:09.105076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41617315-c423-4d16-8e33-a15444797b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.105076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41617315-c423-4d16-8e33-a15444797b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.105076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41617315-c423-4d16-8e33-a15444797b57 admin admin] PATH_INFO: `/v3/projects/03c2fc33ceb84ef5b6a4c5b6027c91b4/users/181bf5b37b81456e908f699c07240175/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.105624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41617315-c423-4d16-8e33-a15444797b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c2fc33ceb84ef5b6a4c5b6027c91b4, user_id=181bf5b37b81456e908f699c07240175, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.108274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d1dfd-0418-471b-9109-7efb5d07b218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:09.108973 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d1dfd-0418-471b-9109-7efb5d07b218 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.109257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d1dfd-0418-471b-9109-7efb5d07b218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.109534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d1dfd-0418-471b-9109-7efb5d07b218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.110291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d1dfd-0418-471b-9109-7efb5d07b218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.110600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41617315-c423-4d16-8e33-a15444797b57 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.111886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d1dfd-0418-471b-9109-7efb5d07b218 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.134447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 811/1633] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:09 2022] PUT /identity/v3/projects/03c2fc33ceb84ef5b6a4c5b6027c91b4/users/181bf5b37b81456e908f699c07240175/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:09.136838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 823/1634] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:09.139501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152940-36ba-47b9-8575-3dc2411325bd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.139653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d1dfd-0418-471b-9109-7efb5d07b218 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.140191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152940-36ba-47b9-8575-3dc2411325bd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.140626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152940-36ba-47b9-8575-3dc2411325bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.156492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d1dfd-0418-471b-9109-7efb5d07b218 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:09.173769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e5152940-36ba-47b9-8575-3dc2411325bd None None] MFA Rules not processed for user `181bf5b37b81456e908f699c07240175`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:09.176309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e5152940-36ba-47b9-8575-3dc2411325bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:09.281775 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232975c5-4a31-48c9-be39-ecd5e14efb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:09.286577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232975c5-4a31-48c9-be39-ecd5e14efb3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.286577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232975c5-4a31-48c9-be39-ecd5e14efb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.286577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232975c5-4a31-48c9-be39-ecd5e14efb3c admin admin] PATH_INFO: `/v3/projects/6b022327bc00410aa8a191c73fbacf8c/users/5dee00f7ded244dcaadfb479e18bb25a/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.287922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 812/1635] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:09.302887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5152940-36ba-47b9-8575-3dc2411325bd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.321706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232975c5-4a31-48c9-be39-ecd5e14efb3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b022327bc00410aa8a191c73fbacf8c, user_id=5dee00f7ded244dcaadfb479e18bb25a, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.329865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232975c5-4a31-48c9-be39-ecd5e14efb3c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.390507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5152940-36ba-47b9-8575-3dc2411325bd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:09.394215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae221327-9366-44e4-aa1b-a6ed6e7879f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:09.394874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221327-9366-44e4-aa1b-a6ed6e7879f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.395155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221327-9366-44e4-aa1b-a6ed6e7879f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.395428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221327-9366-44e4-aa1b-a6ed6e7879f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.409441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae221327-9366-44e4-aa1b-a6ed6e7879f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.415238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae221327-9366-44e4-aa1b-a6ed6e7879f6 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.415388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 824/1636] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:09 2022] PUT /identity/v3/projects/6b022327bc00410aa8a191c73fbacf8c/users/5dee00f7ded244dcaadfb479e18bb25a/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:09.416409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1424f3-928d-4792-bde0-6b440c079ba5 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.417436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1424f3-928d-4792-bde0-6b440c079ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.418012 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1424f3-928d-4792-bde0-6b440c079ba5 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.422634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 813/1637] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:09.426062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b59745-ab27-4a85-871c-8dca99018fec None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.426357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b59745-ab27-4a85-871c-8dca99018fec None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.426633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b59745-ab27-4a85-871c-8dca99018fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.436504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 825/1638] 10.222.0.25 () {54 vars in 912 bytes} [Fri May 13 13:25:09 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:25:09.449418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ed92-3e1d-4741-b84f-038c31c6c6b1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.449912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ed92-3e1d-4741-b84f-038c31c6c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.450662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085ed92-3e1d-4741-b84f-038c31c6c6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.463315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b4b59745-ab27-4a85-871c-8dca99018fec None None] MFA Rules not processed for user `5dee00f7ded244dcaadfb479e18bb25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:09.464540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b4b59745-ab27-4a85-871c-8dca99018fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:09.484889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-8085ed92-3e1d-4741-b84f-038c31c6c6b1 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:09.486250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8085ed92-3e1d-4741-b84f-038c31c6c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:09.579246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 814/1639] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:09.590498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b59745-ab27-4a85-871c-8dca99018fec None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.597699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 826/1640] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:25:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:09.612740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085ed92-3e1d-4741-b84f-038c31c6c6b1 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.686977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b59745-ab27-4a85-871c-8dca99018fec None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:09.693305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfe20e-47ef-42b8-9449-87daff486ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:09.694179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfe20e-47ef-42b8-9449-87daff486ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.694572 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfe20e-47ef-42b8-9449-87daff486ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.694960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfe20e-47ef-42b8-9449-87daff486ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.696029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfe20e-47ef-42b8-9449-87daff486ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.697124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085ed92-3e1d-4741-b84f-038c31c6c6b1 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:09.698546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfe20e-47ef-42b8-9449-87daff486ebe admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.705475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869175f-5aa5-49d3-b766-8c662f447b65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:09.706670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869175f-5aa5-49d3-b766-8c662f447b65 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.707639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869175f-5aa5-49d3-b766-8c662f447b65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.708164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869175f-5aa5-49d3-b766-8c662f447b65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.728934 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869175f-5aa5-49d3-b766-8c662f447b65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.730902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869175f-5aa5-49d3-b766-8c662f447b65 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.760245 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 827/1641] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:09.781546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869175f-5aa5-49d3-b766-8c662f447b65 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.790771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869175f-5aa5-49d3-b766-8c662f447b65 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:09.797689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670dfb4-9c7f-4ff2-8bd3-4c1d4ef975f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:09.805488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670dfb4-9c7f-4ff2-8bd3-4c1d4ef975f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:09.805488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670dfb4-9c7f-4ff2-8bd3-4c1d4ef975f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:09.805488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670dfb4-9c7f-4ff2-8bd3-4c1d4ef975f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:09.878664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 815/1642] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:09 2022] POST /identity/v3/projects => generated 387 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:09.882762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfe20e-47ef-42b8-9449-87daff486ebe admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:09.924061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670dfb4-9c7f-4ff2-8bd3-4c1d4ef975f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:09.926894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670dfb4-9c7f-4ff2-8bd3-4c1d4ef975f8 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:09.940172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 828/1643] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:10.032306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfe20e-47ef-42b8-9449-87daff486ebe admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:10.047902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba869fb3-e91e-4ea6-aedd-35a9e545ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:10.048607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba869fb3-e91e-4ea6-aedd-35a9e545ce64 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:10.048887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba869fb3-e91e-4ea6-aedd-35a9e545ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:10.049177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba869fb3-e91e-4ea6-aedd-35a9e545ce64 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:10.049904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba869fb3-e91e-4ea6-aedd-35a9e545ce64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:10.051872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba869fb3-e91e-4ea6-aedd-35a9e545ce64 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:10.183147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670dfb4-9c7f-4ff2-8bd3-4c1d4ef975f8 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:10.192802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670dfb4-9c7f-4ff2-8bd3-4c1d4ef975f8 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:10.198050 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:10.199571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:10.199859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:10.200133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] PATH_INFO: `/v3/users/b2cef0271718458190ef8c36325c1850` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:10.201894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2cef0271718458190ef8c36325c1850)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:10.203940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:10.211709 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 816/1644] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:09 2022] POST /identity/v3/users => generated 300 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:10.215114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba869fb3-e91e-4ea6-aedd-35a9e545ce64 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:10.226846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba869fb3-e91e-4ea6-aedd-35a9e545ce64 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:10.234209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c718696-0588-40f1-8219-bb2ce2429bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:10.234962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718696-0588-40f1-8219-bb2ce2429bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:10.235545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718696-0588-40f1-8219-bb2ce2429bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:10.235545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718696-0588-40f1-8219-bb2ce2429bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:10.236470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c718696-0588-40f1-8219-bb2ce2429bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:10.238286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c718696-0588-40f1-8219-bb2ce2429bbc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:10.255465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 817/1645] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:10.256465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c718696-0588-40f1-8219-bb2ce2429bbc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:10.269251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c718696-0588-40f1-8219-bb2ce2429bbc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:10.276362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24a14-c48e-453e-bfe0-87f51beb37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:10.277766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24a14-c48e-453e-bfe0-87f51beb37ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:10.277766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24a14-c48e-453e-bfe0-87f51beb37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:10.278023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24a14-c48e-453e-bfe0-87f51beb37ef admin admin] PATH_INFO: `/v3/projects/cf314aaa1bb045edb828a31c17f28777/users/3f8d7466e3724f38b4a86d7720726338/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:10.300223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24a14-c48e-453e-bfe0-87f51beb37ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf314aaa1bb045edb828a31c17f28777, user_id=3f8d7466e3724f38b4a86d7720726338, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:10.304317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24a14-c48e-453e-bfe0-87f51beb37ef admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:10.583713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 818/1646] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:10 2022] PUT /identity/v3/projects/cf314aaa1bb045edb828a31c17f28777/users/3f8d7466e3724f38b4a86d7720726338/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:10.587195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483b1e-451c-4533-bf40-646081a6c1ab None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:10.587195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483b1e-451c-4533-bf40-646081a6c1ab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:10.587195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483b1e-451c-4533-bf40-646081a6c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:10.633217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2cef0271718458190ef8c36325c1850'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:10.634307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2cef0271718458190ef8c36325c1850'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:10.644370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0c483b1e-451c-4533-bf40-646081a6c1ab None None] MFA Rules not processed for user `3755f970272c4d528019826cc7bc73f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:10.645969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0c483b1e-451c-4533-bf40-646081a6c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:10.751316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2cef0271718458190ef8c36325c1850'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:10.760128 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 819/1647] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:10.761299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c483b1e-451c-4533-bf40-646081a6c1ab None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:10.766302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0c483b1e-451c-4533-bf40-646081a6c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:10.770175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2cef0271718458190ef8c36325c1850'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:10.773917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2cef0271718458190ef8c36325c1850'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:10.827279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 829/1648] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:10 2022] DELETE /identity/v3/users/b2cef0271718458190ef8c36325c1850 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:10.829060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:10.831420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:10.897549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c483b1e-451c-4533-bf40-646081a6c1ab None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:10.993295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654e0ae-9e8c-4ca1-a970-e19003406387 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:11.004784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f3dff-e3d8-4142-9bbb-257c05f6fb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.006752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f3dff-e3d8-4142-9bbb-257c05f6fb11 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.007040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f3dff-e3d8-4142-9bbb-257c05f6fb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.007324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f3dff-e3d8-4142-9bbb-257c05f6fb11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.015603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f3dff-e3d8-4142-9bbb-257c05f6fb11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.018746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f3dff-e3d8-4142-9bbb-257c05f6fb11 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.040972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7ddbe-2f9a-47e9-8c21-3e829c4db767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.040972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7ddbe-2f9a-47e9-8c21-3e829c4db767 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.040972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7ddbe-2f9a-47e9-8c21-3e829c4db767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.040972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7ddbe-2f9a-47e9-8c21-3e829c4db767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.041301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7ddbe-2f9a-47e9-8c21-3e829c4db767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.046173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7ddbe-2f9a-47e9-8c21-3e829c4db767 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.062204 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 820/1649] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:11.067565 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57324008-9043-4504-acb3-7313168804a1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.071882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57324008-9043-4504-acb3-7313168804a1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.071882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57324008-9043-4504-acb3-7313168804a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.089608 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-57324008-9043-4504-acb3-7313168804a1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:11.091204 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-57324008-9043-4504-acb3-7313168804a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:11.102689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 821/1650] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:11.103606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57324008-9043-4504-acb3-7313168804a1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:11.115161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57324008-9043-4504-acb3-7313168804a1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:11.128763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 830/1651] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:10 2022] POST /identity/v3/projects => generated 403 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:11.135460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed4f47-0250-4e75-9fa1-bc374b1aeca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.135460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed4f47-0250-4e75-9fa1-bc374b1aeca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.135460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed4f47-0250-4e75-9fa1-bc374b1aeca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.135460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed4f47-0250-4e75-9fa1-bc374b1aeca1 admin admin] PATH_INFO: `/v3/projects/cf314aaa1bb045edb828a31c17f28777/users/3f8d7466e3724f38b4a86d7720726338/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.139817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f3dff-e3d8-4142-9bbb-257c05f6fb11 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:11.151232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed4f47-0250-4e75-9fa1-bc374b1aeca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf314aaa1bb045edb828a31c17f28777, user_id=3f8d7466e3724f38b4a86d7720726338, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.157027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed4f47-0250-4e75-9fa1-bc374b1aeca1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.188806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 822/1652] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:11 2022] PUT /identity/v3/projects/cf314aaa1bb045edb828a31c17f28777/users/3f8d7466e3724f38b4a86d7720726338/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:11.204326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392343d-1536-40d3-939d-b65b17ae79e1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.215479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392343d-1536-40d3-939d-b65b17ae79e1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.215479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392343d-1536-40d3-939d-b65b17ae79e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.267387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8392343d-1536-40d3-939d-b65b17ae79e1 None None] MFA Rules not processed for user `3f8d7466e3724f38b4a86d7720726338`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:11.271038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8392343d-1536-40d3-939d-b65b17ae79e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:11.271913 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f3dff-e3d8-4142-9bbb-257c05f6fb11 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:11.355438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e003f02-5a98-4506-9776-9f32b8d7211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.356128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e003f02-5a98-4506-9776-9f32b8d7211d admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.357040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e003f02-5a98-4506-9776-9f32b8d7211d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.357370 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e003f02-5a98-4506-9776-9f32b8d7211d admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.358189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e003f02-5a98-4506-9776-9f32b8d7211d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.360543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e003f02-5a98-4506-9776-9f32b8d7211d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.384810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 823/1653] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:11.386017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392343d-1536-40d3-939d-b65b17ae79e1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:11.388427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8392343d-1536-40d3-939d-b65b17ae79e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:11.464631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392343d-1536-40d3-939d-b65b17ae79e1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:11.469614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 831/1654] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:11 2022] POST /identity/v3/users => generated 308 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:11.470930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b45e6-daea-4bae-9129-dc60ef696584 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.471218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b45e6-daea-4bae-9129-dc60ef696584 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.471571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b45e6-daea-4bae-9129-dc60ef696584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.474061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.474866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.475269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.475635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.494494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.496335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.713230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 824/1655] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:11.713230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:11.713230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:11.716845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-838b45e6-daea-4bae-9129-dc60ef696584 None None] MFA Rules not processed for user `80623f3527844489a5e31a786080fa10`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:11.716845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-838b45e6-daea-4bae-9129-dc60ef696584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:11.717366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4408f43-9c89-44ba-92a3-ac8ae5ded320 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:11.717366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67738f-3d80-431d-a790-fb68cab22c1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.717366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67738f-3d80-431d-a790-fb68cab22c1f service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.717366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67738f-3d80-431d-a790-fb68cab22c1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.717985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67738f-3d80-431d-a790-fb68cab22c1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.717985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67738f-3d80-431d-a790-fb68cab22c1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.718250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 832/1656] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:11.718250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b45e6-daea-4bae-9129-dc60ef696584 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:11.718502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67738f-3d80-431d-a790-fb68cab22c1f service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.718502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 825/1657] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:11.718502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45762b2-244e-46ae-9b69-8bcd4d46cc2e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.718502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45762b2-244e-46ae-9b69-8bcd4d46cc2e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.718502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45762b2-244e-46ae-9b69-8bcd4d46cc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.718502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a45762b2-244e-46ae-9b69-8bcd4d46cc2e None None] MFA Rules not processed for user `80623f3527844489a5e31a786080fa10`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:11.720919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a45762b2-244e-46ae-9b69-8bcd4d46cc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:11.720919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 826/1658] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:11.720919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45762b2-244e-46ae-9b69-8bcd4d46cc2e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:11.720919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a45762b2-244e-46ae-9b69-8bcd4d46cc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:11.721387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b45e6-daea-4bae-9129-dc60ef696584 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:11.727915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608259b5-9937-4fc9-92b1-d81de57cea67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.729442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608259b5-9937-4fc9-92b1-d81de57cea67 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.730569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608259b5-9937-4fc9-92b1-d81de57cea67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.730569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608259b5-9937-4fc9-92b1-d81de57cea67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.732446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608259b5-9937-4fc9-92b1-d81de57cea67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.735298 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608259b5-9937-4fc9-92b1-d81de57cea67 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.761383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 833/1659] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:11.764496 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608259b5-9937-4fc9-92b1-d81de57cea67 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:11.790933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608259b5-9937-4fc9-92b1-d81de57cea67 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:11.802119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052f7e-7f83-496c-92cc-b8beb1a74046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.803197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052f7e-7f83-496c-92cc-b8beb1a74046 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.803311 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052f7e-7f83-496c-92cc-b8beb1a74046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.803585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052f7e-7f83-496c-92cc-b8beb1a74046 admin admin] PATH_INFO: `/v3/projects/3d97dea4cb5349bcb7d66e156426c913/users/05165a37427143bb91e21f3d794425fe/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.835513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45762b2-244e-46ae-9b69-8bcd4d46cc2e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:11.847157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0052f7e-7f83-496c-92cc-b8beb1a74046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d97dea4cb5349bcb7d66e156426c913, user_id=05165a37427143bb91e21f3d794425fe, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.847448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08a39d-0528-4359-b944-47def9d773c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:11.847448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08a39d-0528-4359-b944-47def9d773c2 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:11.847448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08a39d-0528-4359-b944-47def9d773c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:11.847448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08a39d-0528-4359-b944-47def9d773c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:11.848912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0052f7e-7f83-496c-92cc-b8beb1a74046 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.861577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08a39d-0528-4359-b944-47def9d773c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:11.870240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08a39d-0528-4359-b944-47def9d773c2 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:11.875954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 827/1660] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:11.955635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 834/1661] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:11 2022] PUT /identity/v3/projects/3d97dea4cb5349bcb7d66e156426c913/users/05165a37427143bb91e21f3d794425fe/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:11.960761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08a39d-0528-4359-b944-47def9d773c2 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:11.978202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08a39d-0528-4359-b944-47def9d773c2 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:12.112714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f65b4-6a1c-4cdb-90ef-15336eeaffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:12.112714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f65b4-6a1c-4cdb-90ef-15336eeaffb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:12.112714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f65b4-6a1c-4cdb-90ef-15336eeaffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:12.112714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f65b4-6a1c-4cdb-90ef-15336eeaffb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:12.113648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f65b4-6a1c-4cdb-90ef-15336eeaffb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:12.117470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f65b4-6a1c-4cdb-90ef-15336eeaffb0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:12.147935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 828/1662] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:12.151902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052f7e-7f83-496c-92cc-b8beb1a74046 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:12.166998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052f7e-7f83-496c-92cc-b8beb1a74046 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:12.180075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb454b24-972c-4dc5-9b3a-0ba5fd9a52a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:12.181071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb454b24-972c-4dc5-9b3a-0ba5fd9a52a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:12.181366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb454b24-972c-4dc5-9b3a-0ba5fd9a52a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:12.187103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb454b24-972c-4dc5-9b3a-0ba5fd9a52a2 admin admin] PATH_INFO: `/v3/projects/3d97dea4cb5349bcb7d66e156426c913/users/05165a37427143bb91e21f3d794425fe/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:12.187917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb454b24-972c-4dc5-9b3a-0ba5fd9a52a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d97dea4cb5349bcb7d66e156426c913, user_id=05165a37427143bb91e21f3d794425fe, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:12.204470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb454b24-972c-4dc5-9b3a-0ba5fd9a52a2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:12.234293 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 835/1663] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:12 2022] PUT /identity/v3/projects/3d97dea4cb5349bcb7d66e156426c913/users/05165a37427143bb91e21f3d794425fe/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:12.714887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4a6be-0200-49d1-85d7-be95b9f12902 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:12.715384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4a6be-0200-49d1-85d7-be95b9f12902 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:12.715458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4a6be-0200-49d1-85d7-be95b9f12902 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:12.717060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 829/1664] 10.222.0.25 () {54 vars in 912 bytes} [Fri May 13 13:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:25:12.725495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04cf0f-24df-43e2-8044-b324c88d666a None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:12.726871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04cf0f-24df-43e2-8044-b324c88d666a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:12.727375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04cf0f-24df-43e2-8044-b324c88d666a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:12.788766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1f04cf0f-24df-43e2-8044-b324c88d666a None None] MFA Rules not processed for user `8ddb0dcaa5304016af1bc1f88845fb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:12.789412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1f04cf0f-24df-43e2-8044-b324c88d666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:12.848200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 836/1665] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:12.859662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4a6be-0200-49d1-85d7-be95b9f12902 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:12.877484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4a6be-0200-49d1-85d7-be95b9f12902 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:12.877873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0518c-5a96-4bd4-ad12-552cd8cfdab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:12.878523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0518c-5a96-4bd4-ad12-552cd8cfdab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:12.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0518c-5a96-4bd4-ad12-552cd8cfdab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:12.879077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0518c-5a96-4bd4-ad12-552cd8cfdab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:12.886652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0518c-5a96-4bd4-ad12-552cd8cfdab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:12.888654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0518c-5a96-4bd4-ad12-552cd8cfdab2 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:12.900112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 830/1666] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:13.082430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04cf0f-24df-43e2-8044-b324c88d666a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:13.086141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1f04cf0f-24df-43e2-8044-b324c88d666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:13.244594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04cf0f-24df-43e2-8044-b324c88d666a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:13.257877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:13.263585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:13.263585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:13.263585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] PATH_INFO: `/v3/projects/942433b7b354400a95fdfe5614773495` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:13.264993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=942433b7b354400a95fdfe5614773495)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:13.266453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: WARNING py.warnings [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:25:13.266453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: warnings.warn(msg) May 13 13:25:13.266453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: May 13 13:25:13.268150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:13.585131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942433b7b354400a95fdfe5614773495', 'request_id': 'req-5228d863-3884-4fec-861a-34e95e53ef9c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:13.616643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942433b7b354400a95fdfe5614773495', 'request_id': 'req-5228d863-3884-4fec-861a-34e95e53ef9c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:13.721271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942433b7b354400a95fdfe5614773495', 'request_id': 'req-5228d863-3884-4fec-861a-34e95e53ef9c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:13.768150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] The token cache is being invalidate because project 942433b7b354400a95fdfe5614773495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:13.768458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:13.770269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:13.774444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 837/1667] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:13 2022] DELETE /identity/v3/projects/942433b7b354400a95fdfe5614773495 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:13.779295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd02a-1e92-4dae-945e-657952bc9d11 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:13.780275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd02a-1e92-4dae-945e-657952bc9d11 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:13.781104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd02a-1e92-4dae-945e-657952bc9d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:13.790001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:13.791490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:13.826936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-98dfd02a-1e92-4dae-945e-657952bc9d11 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:13.834585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-98dfd02a-1e92-4dae-945e-657952bc9d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:13.925770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228d863-3884-4fec-861a-34e95e53ef9c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:13.936625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:13.937454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:13.941011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:13.942626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:13.947159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:14.020194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 831/1668] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:14.024996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c4b95-3be7-4e2a-9e05-b5a0f69d0b92 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.026652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c4b95-3be7-4e2a-9e05-b5a0f69d0b92 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.028950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c4b95-3be7-4e2a-9e05-b5a0f69d0b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.074522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8a7c4b95-3be7-4e2a-9e05-b5a0f69d0b92 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:14.078339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8a7c4b95-3be7-4e2a-9e05-b5a0f69d0b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:14.087102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:14.092071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:14.109600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 838/1669] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:14.134927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 832/1670] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:14.139682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:14.161942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acfb7a-afc5-4e6f-be73-8da8652906cb service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:14.172683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e56c0-b26b-4b48-9562-45b776e86d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:14.172683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e56c0-b26b-4b48-9562-45b776e86d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.174616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e56c0-b26b-4b48-9562-45b776e86d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.174616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e56c0-b26b-4b48-9562-45b776e86d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.175672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e56c0-b26b-4b48-9562-45b776e86d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:14.176131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e56c0-b26b-4b48-9562-45b776e86d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:14.179595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e56c0-b26b-4b48-9562-45b776e86d36 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:14.235861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 839/1671] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:25:14.241233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c4b95-3be7-4e2a-9e05-b5a0f69d0b92 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:14.259650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c4b95-3be7-4e2a-9e05-b5a0f69d0b92 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:14.262542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a870b-28f0-4b1b-b4ff-81dbc650b324 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.263614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a870b-28f0-4b1b-b4ff-81dbc650b324 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.264082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a870b-28f0-4b1b-b4ff-81dbc650b324 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.266020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 840/1672] 10.222.0.25 () {54 vars in 960 bytes} [Fri May 13 13:25:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:25:14.271828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169a898-f87a-43e7-a3a0-d8f27fce0dcf None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.272210 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169a898-f87a-43e7-a3a0-d8f27fce0dcf None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.275279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169a898-f87a-43e7-a3a0-d8f27fce0dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.275390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99ffe3-f99a-4205-a845-5b2df322e0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:14.275390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99ffe3-f99a-4205-a845-5b2df322e0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.275390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99ffe3-f99a-4205-a845-5b2df322e0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.275390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99ffe3-f99a-4205-a845-5b2df322e0e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.275942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99ffe3-f99a-4205-a845-5b2df322e0e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:14.280624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99ffe3-f99a-4205-a845-5b2df322e0e7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:14.310193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-3169a898-f87a-43e7-a3a0-d8f27fce0dcf None None] MFA Rules not processed for user `310c5d3ba81044ac8f3f192728cc3e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:14.311813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3169a898-f87a-43e7-a3a0-d8f27fce0dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:14.393568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 833/1673] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:14 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:14.397901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99ffe3-f99a-4205-a845-5b2df322e0e7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:14.461007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 841/1674] 10.222.0.25 () {58 vars in 1057 bytes} [Fri May 13 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 13 13:25:14.462416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169a898-f87a-43e7-a3a0-d8f27fce0dcf None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:14.491965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169a898-f87a-43e7-a3a0-d8f27fce0dcf None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:14.589614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99ffe3-f99a-4205-a845-5b2df322e0e7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:14.599479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fffb2b-a380-4b2a-8840-fc8e052a3210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:14.599479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fffb2b-a380-4b2a-8840-fc8e052a3210 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.599479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fffb2b-a380-4b2a-8840-fc8e052a3210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.599479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fffb2b-a380-4b2a-8840-fc8e052a3210 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.599787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fffb2b-a380-4b2a-8840-fc8e052a3210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:14.601556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fffb2b-a380-4b2a-8840-fc8e052a3210 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:14.658130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '310c5d3ba81044ac8f3f192728cc3e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:14.664549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.664549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.664549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.664549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:14.694863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 834/1675] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:14 2022] POST /identity/v3/users => generated 286 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:14.696717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fffb2b-a380-4b2a-8840-fc8e052a3210 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:14.697987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:14.700833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:14.729231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 842/1676] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:14.742611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fffb2b-a380-4b2a-8840-fc8e052a3210 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:14.756391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab6088-47f7-4536-b7a8-588481af110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:14.756810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab6088-47f7-4536-b7a8-588481af110c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.757156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab6088-47f7-4536-b7a8-588481af110c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.758254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab6088-47f7-4536-b7a8-588481af110c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.759002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab6088-47f7-4536-b7a8-588481af110c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:14.760634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab6088-47f7-4536-b7a8-588481af110c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:14.794064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 835/1677] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:14.800044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:14.848773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67e1b0-6575-4d74-8eca-e017a2b58a3c service placement] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:14.859760 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2f89b-5833-4522-a3a1-2692f8e70aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:14.863043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2f89b-5833-4522-a3a1-2692f8e70aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.864586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2f89b-5833-4522-a3a1-2692f8e70aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.865462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2f89b-5833-4522-a3a1-2692f8e70aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.869441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2f89b-5833-4522-a3a1-2692f8e70aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:14.873458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2f89b-5833-4522-a3a1-2692f8e70aa1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:14.891621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 843/1678] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:14.895748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab6088-47f7-4536-b7a8-588481af110c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:14.908758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab6088-47f7-4536-b7a8-588481af110c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:14.930583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1ba2-3ce6-45c6-9805-3efb0389e303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:14.931656 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1ba2-3ce6-45c6-9805-3efb0389e303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:14.931953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1ba2-3ce6-45c6-9805-3efb0389e303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:14.933390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1ba2-3ce6-45c6-9805-3efb0389e303 admin admin] PATH_INFO: `/v3/projects/4c9872b9ac4647c4bb98bc8ec82a6ca5/users/e0cd13c6d5a540bbb8dd02bcfedeb211/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:14.994297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d1ba2-3ce6-45c6-9805-3efb0389e303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9872b9ac4647c4bb98bc8ec82a6ca5, user_id=e0cd13c6d5a540bbb8dd02bcfedeb211, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:15.005835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d1ba2-3ce6-45c6-9805-3efb0389e303 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:15.127146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 836/1679] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:14 2022] PUT /identity/v3/projects/4c9872b9ac4647c4bb98bc8ec82a6ca5/users/e0cd13c6d5a540bbb8dd02bcfedeb211/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:15.136694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9ba1-3de1-4b3a-989a-3a1c0f00a014 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:15.137216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9ba1-3de1-4b3a-989a-3a1c0f00a014 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:15.137316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9ba1-3de1-4b3a-989a-3a1c0f00a014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:15.191455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-3edd9ba1-3de1-4b3a-989a-3a1c0f00a014 None None] MFA Rules not processed for user `e0cd13c6d5a540bbb8dd02bcfedeb211`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:15.192693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3edd9ba1-3de1-4b3a-989a-3a1c0f00a014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:15.429667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 844/1680] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:15.431151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1ba2-3ce6-45c6-9805-3efb0389e303 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:15.516360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1ba2-3ce6-45c6-9805-3efb0389e303 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:15.525529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639de3a-0428-45a7-9b01-866b6b657b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:15.527282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639de3a-0428-45a7-9b01-866b6b657b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:15.527733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639de3a-0428-45a7-9b01-866b6b657b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:15.528775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639de3a-0428-45a7-9b01-866b6b657b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:15.529631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639de3a-0428-45a7-9b01-866b6b657b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:15.531728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639de3a-0428-45a7-9b01-866b6b657b53 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:15.639893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 837/1681] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:15 2022] POST /identity/v3/projects => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:15.641842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9ba1-3de1-4b3a-989a-3a1c0f00a014 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:15.665883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9ba1-3de1-4b3a-989a-3a1c0f00a014 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:15.857665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f05d5f-50ee-4a5c-8a32-c2c0ae7f4f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:15.858417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f05d5f-50ee-4a5c-8a32-c2c0ae7f4f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:15.858709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f05d5f-50ee-4a5c-8a32-c2c0ae7f4f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:15.858988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f05d5f-50ee-4a5c-8a32-c2c0ae7f4f1e admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:15.860452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f05d5f-50ee-4a5c-8a32-c2c0ae7f4f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:15.865493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f05d5f-50ee-4a5c-8a32-c2c0ae7f4f1e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:15.991283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 845/1682] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:15 2022] POST /identity/v3/users => generated 293 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:15.994997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639de3a-0428-45a7-9b01-866b6b657b53 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:16.014889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639de3a-0428-45a7-9b01-866b6b657b53 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:16.024696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ed8d-2514-4474-8029-8932ed50e78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:16.025996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ed8d-2514-4474-8029-8932ed50e78f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:16.045734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ed8d-2514-4474-8029-8932ed50e78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:16.045734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ed8d-2514-4474-8029-8932ed50e78f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:16.045734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8ed8d-2514-4474-8029-8932ed50e78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:16.045734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8ed8d-2514-4474-8029-8932ed50e78f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:16.067424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 838/1683] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:16.071892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f05d5f-50ee-4a5c-8a32-c2c0ae7f4f1e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:16.089096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f05d5f-50ee-4a5c-8a32-c2c0ae7f4f1e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:16.113820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0328ed-ddc3-467a-bc3e-bea573c7c532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:16.116606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0328ed-ddc3-467a-bc3e-bea573c7c532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:16.118510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0328ed-ddc3-467a-bc3e-bea573c7c532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:16.119404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0328ed-ddc3-467a-bc3e-bea573c7c532 admin admin] PATH_INFO: `/v3/projects/42cb0eaf89b94e948fdbc3862018b5e7/users/54760dc7acdd4ce6934b402d2bc7dc47/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:16.190431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0328ed-ddc3-467a-bc3e-bea573c7c532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cb0eaf89b94e948fdbc3862018b5e7, user_id=54760dc7acdd4ce6934b402d2bc7dc47, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:16.195500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0328ed-ddc3-467a-bc3e-bea573c7c532 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:16.367857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 846/1684] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:16 2022] PUT /identity/v3/projects/42cb0eaf89b94e948fdbc3862018b5e7/users/54760dc7acdd4ce6934b402d2bc7dc47/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:16.374180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ed8d-2514-4474-8029-8932ed50e78f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:16.415733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ed8d-2514-4474-8029-8932ed50e78f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:16.528555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f76c5-82a2-49fe-80c0-043019e0e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:16.529536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f76c5-82a2-49fe-80c0-043019e0e84f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:16.530122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f76c5-82a2-49fe-80c0-043019e0e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:16.530750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f76c5-82a2-49fe-80c0-043019e0e84f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:16.531696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f76c5-82a2-49fe-80c0-043019e0e84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:16.533634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f76c5-82a2-49fe-80c0-043019e0e84f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:16.556606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0328ed-ddc3-467a-bc3e-bea573c7c532 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:16.565518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 839/1685] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:16.569803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0328ed-ddc3-467a-bc3e-bea573c7c532 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:16.579366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1e3c-93ea-4db1-8aef-d5c8d97c8432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:16.580836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1e3c-93ea-4db1-8aef-d5c8d97c8432 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:16.581193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1e3c-93ea-4db1-8aef-d5c8d97c8432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:16.581579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1e3c-93ea-4db1-8aef-d5c8d97c8432 admin admin] PATH_INFO: `/v3/projects/42cb0eaf89b94e948fdbc3862018b5e7/users/54760dc7acdd4ce6934b402d2bc7dc47/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:16.584496 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc1e3c-93ea-4db1-8aef-d5c8d97c8432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cb0eaf89b94e948fdbc3862018b5e7, user_id=54760dc7acdd4ce6934b402d2bc7dc47, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:16.591216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc1e3c-93ea-4db1-8aef-d5c8d97c8432 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:16.619759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa0452-bf79-449a-963b-853311a1ca21 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:16.620055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa0452-bf79-449a-963b-853311a1ca21 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:16.620788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa0452-bf79-449a-963b-853311a1ca21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:16.624103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 847/1686] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:16 2022] PUT /identity/v3/projects/42cb0eaf89b94e948fdbc3862018b5e7/users/54760dc7acdd4ce6934b402d2bc7dc47/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:16.659876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1daa0452-bf79-449a-963b-853311a1ca21 None None] MFA Rules not processed for user `54760dc7acdd4ce6934b402d2bc7dc47`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:16.664755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1daa0452-bf79-449a-963b-853311a1ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:16.826246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 840/1687] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:16.860877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1e3c-93ea-4db1-8aef-d5c8d97c8432 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:16.946950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1e3c-93ea-4db1-8aef-d5c8d97c8432 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:16.951029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509e98-9718-4bf8-b54a-fc40b77470db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:16.951764 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509e98-9718-4bf8-b54a-fc40b77470db service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:16.952114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509e98-9718-4bf8-b54a-fc40b77470db service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:16.952492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509e98-9718-4bf8-b54a-fc40b77470db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:16.963038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b509e98-9718-4bf8-b54a-fc40b77470db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:16.965294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b509e98-9718-4bf8-b54a-fc40b77470db service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:16.983418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6130-3c8a-490a-94c9-57e1c5bf8cf9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:16.983708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6130-3c8a-490a-94c9-57e1c5bf8cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:16.985182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6130-3c8a-490a-94c9-57e1c5bf8cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:17.000787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 848/1688] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:17.052671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-509b6130-3c8a-490a-94c9-57e1c5bf8cf9 None None] MFA Rules not processed for user `05165a37427143bb91e21f3d794425fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:17.052671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-509b6130-3c8a-490a-94c9-57e1c5bf8cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:17.256720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 841/1689] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:17.364111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509e98-9718-4bf8-b54a-fc40b77470db service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:17.375349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509e98-9718-4bf8-b54a-fc40b77470db service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:17.390229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:17.395967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:17.395967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:17.395967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:17.531041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:17.531859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:17.565926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 849/1690] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:20.353956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8a1bb-d729-484c-919b-91d3e9d3518e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:20.356632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8a1bb-d729-484c-919b-91d3e9d3518e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:20.356632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8a1bb-d729-484c-919b-91d3e9d3518e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:20.413588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-39f8a1bb-d729-484c-919b-91d3e9d3518e None None] MFA Rules not processed for user `08b9fbf87cfa417fbea857ae602a140e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:20.415110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-39f8a1bb-d729-484c-919b-91d3e9d3518e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:20.594186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 842/1691] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:20.609755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:20.612232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:20.798689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a3ec3-08df-424c-8d1c-e409a0d91556 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:20.811216 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:20.811894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:20.812177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:20.812453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:20.840641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:20.844077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:20.867814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 850/1692] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:22.451170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0e910-7a81-4c3d-a310-fbf334faf6ab None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:22.451735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0e910-7a81-4c3d-a310-fbf334faf6ab None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:22.451883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0e910-7a81-4c3d-a310-fbf334faf6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:22.492646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-31c0e910-7a81-4c3d-a310-fbf334faf6ab None None] MFA Rules not processed for user `f56c908ada31445e85ec1dfee8e2ebdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:22.498006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-31c0e910-7a81-4c3d-a310-fbf334faf6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:22.617286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:22.619299 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:22.683767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 843/1693] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:22.696234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841a9ca-64be-4beb-8a5b-d9499f09dec9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:22.697010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841a9ca-64be-4beb-8a5b-d9499f09dec9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:22.697010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841a9ca-64be-4beb-8a5b-d9499f09dec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:22.749421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d841a9ca-64be-4beb-8a5b-d9499f09dec9 None None] MFA Rules not processed for user `f56c908ada31445e85ec1dfee8e2ebdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:22.751615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d841a9ca-64be-4beb-8a5b-d9499f09dec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:22.776889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713a48-71cb-49f5-8e83-d06bae20e2a7 service nova] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:22.785333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 844/1694] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:22.786884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:22.788160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:22.788628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:22.789176 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] PATH_INFO: `/v3/users/3755f970272c4d528019826cc7bc73f1` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:22.791527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3755f970272c4d528019826cc7bc73f1)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:22.798308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:22.816466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841a9ca-64be-4beb-8a5b-d9499f09dec9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:22.917130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841a9ca-64be-4beb-8a5b-d9499f09dec9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:22.939493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c55fd-58a7-47b4-8346-da5e2a484ae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:22.942369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c55fd-58a7-47b4-8346-da5e2a484ae3 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:22.942889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c55fd-58a7-47b4-8346-da5e2a484ae3 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:22.943341 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c55fd-58a7-47b4-8346-da5e2a484ae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:22.986369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c55fd-58a7-47b4-8346-da5e2a484ae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:22.996366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c55fd-58a7-47b4-8346-da5e2a484ae3 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:23.074550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 845/1695] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:23.082301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81095d-f05b-4e31-a8cf-62ef0ca60037 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:23.086835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81095d-f05b-4e31-a8cf-62ef0ca60037 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:23.087462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81095d-f05b-4e31-a8cf-62ef0ca60037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:23.127914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5a81095d-f05b-4e31-a8cf-62ef0ca60037 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:23.129322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5a81095d-f05b-4e31-a8cf-62ef0ca60037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:23.236761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 846/1696] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:25:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:23.237888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81095d-f05b-4e31-a8cf-62ef0ca60037 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:23.239805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5a81095d-f05b-4e31-a8cf-62ef0ca60037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:23.278784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81095d-f05b-4e31-a8cf-62ef0ca60037 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:23.294945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d77ab-497c-49ab-9517-224cd91e6d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:23.294945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d77ab-497c-49ab-9517-224cd91e6d4e service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:23.295171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d77ab-497c-49ab-9517-224cd91e6d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:23.295378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d77ab-497c-49ab-9517-224cd91e6d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:23.303249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9b7d77ab-497c-49ab-9517-224cd91e6d4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:23.364049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3755f970272c4d528019826cc7bc73f1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:23.365104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3755f970272c4d528019826cc7bc73f1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:23.489809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3755f970272c4d528019826cc7bc73f1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:23.495386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d77ab-497c-49ab-9517-224cd91e6d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:23.499902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3755f970272c4d528019826cc7bc73f1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:23.501947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3755f970272c4d528019826cc7bc73f1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:23.503000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d77ab-497c-49ab-9517-224cd91e6d4e service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:23.517742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 847/1697] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:23.542649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 851/1698] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:22 2022] DELETE /identity/v3/users/3755f970272c4d528019826cc7bc73f1 => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:23.794083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d0d62-ee3d-4f2a-9a28-56c16d5718da None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:23.794568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d0d62-ee3d-4f2a-9a28-56c16d5718da None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:23.795379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d0d62-ee3d-4f2a-9a28-56c16d5718da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:23.844801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:23.849385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:23.880476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2d6d0d62-ee3d-4f2a-9a28-56c16d5718da None None] MFA Rules not processed for user `233f5342cf334c7689cb9e71f4fbaab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:23.885500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2d6d0d62-ee3d-4f2a-9a28-56c16d5718da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:24.031307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 848/1699] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:24.031525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8ec6e-21a5-4ee2-8ac6-caebfdcfe502 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:24.050832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d0d62-ee3d-4f2a-9a28-56c16d5718da None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:24.052674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2d6d0d62-ee3d-4f2a-9a28-56c16d5718da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:24.052781 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:24.053396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:24.053687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:24.053997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:24.060852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:24.176009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d0d62-ee3d-4f2a-9a28-56c16d5718da None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:24.193070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b443d5-e002-40b5-a583-5182f38cbd1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:24.194990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b443d5-e002-40b5-a583-5182f38cbd1b service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:24.194990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b443d5-e002-40b5-a583-5182f38cbd1b service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:24.194990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b443d5-e002-40b5-a583-5182f38cbd1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:24.215144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b443d5-e002-40b5-a583-5182f38cbd1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:24.220060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b443d5-e002-40b5-a583-5182f38cbd1b service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:24.245859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:24.252249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:24.275264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 852/1700] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:25:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:25:24.294235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 849/1701] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:24.984803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:24.987197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:25.131914 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4497e-803f-4516-bd85-e1095fc2a3ec service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:25.138793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:25.139698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:25.140055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:25.140321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] PATH_INFO: `/v3/projects/27638a23724b420680f3287c88580d31` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:25.142606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27638a23724b420680f3287c88580d31)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:25.144457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:25.413195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27638a23724b420680f3287c88580d31', 'request_id': 'req-2552d381-467d-4dd3-b74b-f57a54185808'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:25.436831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27638a23724b420680f3287c88580d31', 'request_id': 'req-2552d381-467d-4dd3-b74b-f57a54185808'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:25.589172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27638a23724b420680f3287c88580d31', 'request_id': 'req-2552d381-467d-4dd3-b74b-f57a54185808'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:25.636000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] The token cache is being invalidate because project 27638a23724b420680f3287c88580d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:25.638168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:25.641385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2552d381-467d-4dd3-b74b-f57a54185808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:25.650864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 853/1702] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:24 2022] DELETE /identity/v3/projects/27638a23724b420680f3287c88580d31 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:25.655966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b443d5-e002-40b5-a583-5182f38cbd1b service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:25.684343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b443d5-e002-40b5-a583-5182f38cbd1b service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:25.813049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:25.814016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:25.814307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:25.814590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] PATH_INFO: `/v3/users/05165a37427143bb91e21f3d794425fe` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:25.815883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=05165a37427143bb91e21f3d794425fe)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:25.817087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: WARNING py.warnings [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 13 13:25:25.817087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: warnings.warn(msg) May 13 13:25:25.817087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: May 13 13:25:25.819148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:26.120440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53774-fb18-4e33-b2ee-fda8f6fc1cb4 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:26.120440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53774-fb18-4e33-b2ee-fda8f6fc1cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:26.120440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53774-fb18-4e33-b2ee-fda8f6fc1cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:26.120440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bdc53774-fb18-4e33-b2ee-fda8f6fc1cb4 None None] MFA Rules not processed for user `f56c908ada31445e85ec1dfee8e2ebdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:26.120440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bdc53774-fb18-4e33-b2ee-fda8f6fc1cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:26.120440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 854/1703] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:27.941742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05165a37427143bb91e21f3d794425fe'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:27.948368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05165a37427143bb91e21f3d794425fe'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:27.951342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05165a37427143bb91e21f3d794425fe'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:27.998767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05165a37427143bb91e21f3d794425fe'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:28.153703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3351f7-dc71-4d43-a619-b23c18da8b83 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:28.154390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3351f7-dc71-4d43-a619-b23c18da8b83 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:28.154579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3351f7-dc71-4d43-a619-b23c18da8b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:28.226738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2a3351f7-dc71-4d43-a619-b23c18da8b83 None None] MFA Rules not processed for user `8ddb0dcaa5304016af1bc1f88845fb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:28.228777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2a3351f7-dc71-4d43-a619-b23c18da8b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:28.485382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 855/1704] 10.222.0.70 () {58 vars in 1015 bytes} [Fri May 13 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:28.498690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3351f7-dc71-4d43-a619-b23c18da8b83 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:28.500801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2a3351f7-dc71-4d43-a619-b23c18da8b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:28.531845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3351f7-dc71-4d43-a619-b23c18da8b83 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:28.552272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:28.562710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:28.563004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:28.563290 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:28.605819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:28.615035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:28.655870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 856/1705] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:28.729189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b29a94ad-6301-49eb-92d7-323db8903d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05165a37427143bb91e21f3d794425fe'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:28.811737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 850/1706] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:25 2022] DELETE /identity/v3/users/05165a37427143bb91e21f3d794425fe => generated 0 bytes in 3157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:29.806572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:29.810786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:29.851343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb6d77-9646-474f-8d08-0b2d310618ee service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:30.094202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:30.098623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:30.111127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:30.112162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] PATH_INFO: `/v3/users/64db0dfebf94402d978d9b0d0639a695` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:30.127670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64db0dfebf94402d978d9b0d0639a695)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:30.142348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:30.325798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28b612-e5a3-4092-a553-0fbe6134ebb0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:30.326422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28b612-e5a3-4092-a553-0fbe6134ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:30.327051 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28b612-e5a3-4092-a553-0fbe6134ebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:30.374038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1b28b612-e5a3-4092-a553-0fbe6134ebb0 None None] MFA Rules not processed for user `08b9fbf87cfa417fbea857ae602a140e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:30.382181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1b28b612-e5a3-4092-a553-0fbe6134ebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:30.478847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64db0dfebf94402d978d9b0d0639a695'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:30.481842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64db0dfebf94402d978d9b0d0639a695'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:30.604863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64db0dfebf94402d978d9b0d0639a695'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:30.616337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64db0dfebf94402d978d9b0d0639a695'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:30.618586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a80e0ab0-c63d-4d0b-bc9b-5e2cd35793b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64db0dfebf94402d978d9b0d0639a695'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:30.636682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 851/1707] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:30.642687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28b612-e5a3-4092-a553-0fbe6134ebb0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:30.650484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1b28b612-e5a3-4092-a553-0fbe6134ebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:30.708432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28b612-e5a3-4092-a553-0fbe6134ebb0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:30.718553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:30.724309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 857/1708] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:29 2022] DELETE /identity/v3/users/64db0dfebf94402d978d9b0d0639a695 => generated 0 bytes in 918 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:30.729706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:30.729895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c447b-f919-4b67-9392-dfb957bbc4d6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:30.732003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:30.733488 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] PATH_INFO: `/v3/projects/3d97dea4cb5349bcb7d66e156426c913` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:30.735422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c447b-f919-4b67-9392-dfb957bbc4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:30.735793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c447b-f919-4b67-9392-dfb957bbc4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:30.738890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d97dea4cb5349bcb7d66e156426c913)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:30.744469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:30.791641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-c46c447b-f919-4b67-9392-dfb957bbc4d6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:30.795226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c46c447b-f919-4b67-9392-dfb957bbc4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:31.101378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 858/1709] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:31.159118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c447b-f919-4b67-9392-dfb957bbc4d6 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:31.161517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c46c447b-f919-4b67-9392-dfb957bbc4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:31.165150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d97dea4cb5349bcb7d66e156426c913', 'request_id': 'req-22cec5e9-7725-4926-b414-61e8664e3bca'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:31.180207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d97dea4cb5349bcb7d66e156426c913', 'request_id': 'req-22cec5e9-7725-4926-b414-61e8664e3bca'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:31.302322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d97dea4cb5349bcb7d66e156426c913', 'request_id': 'req-22cec5e9-7725-4926-b414-61e8664e3bca'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:31.334596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] The token cache is being invalidate because project 3d97dea4cb5349bcb7d66e156426c913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:31.335271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:31.339253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22cec5e9-7725-4926-b414-61e8664e3bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:31.341637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 852/1710] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:30 2022] DELETE /identity/v3/projects/3d97dea4cb5349bcb7d66e156426c913 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:31.348020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad182-b785-44ec-90f2-83b81a362084 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:31.348564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad182-b785-44ec-90f2-83b81a362084 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:31.349642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad182-b785-44ec-90f2-83b81a362084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:31.375750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c447b-f919-4b67-9392-dfb957bbc4d6 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:31.420679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-acbad182-b785-44ec-90f2-83b81a362084 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:31.424008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-acbad182-b785-44ec-90f2-83b81a362084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:31.599972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:31.601137 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:31.604184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:31.605036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:31.611347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:31.675918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 853/1711] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:31.692114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1a42-92c2-4644-b7cd-6f19ad7766ef None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:31.692532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1a42-92c2-4644-b7cd-6f19ad7766ef None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:31.693296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e1a42-92c2-4644-b7cd-6f19ad7766ef None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:31.700866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 854/1712] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:25:31 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:25:31.702622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c1ef-53c1-412b-9608-9369725e63a3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:31.704567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c1ef-53c1-412b-9608-9369725e63a3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:31.705631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c1ef-53c1-412b-9608-9369725e63a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:31.754190 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e655c1ef-53c1-412b-9608-9369725e63a3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:31.757545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e655c1ef-53c1-412b-9608-9369725e63a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:31.797116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 855/1713] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:31.797116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc68f-c024-4d93-bc06-23c5f0fa84c7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:31.797116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc68f-c024-4d93-bc06-23c5f0fa84c7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:31.797116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc68f-c024-4d93-bc06-23c5f0fa84c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:31.839355 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:31.851709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:31.884560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 859/1714] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:31.885476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:31.914506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee175c7-f38d-4385-943f-f3e3abacef0e service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:31.914997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4d1bc68f-c024-4d93-bc06-23c5f0fa84c7 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:31.930679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:31.931504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:31.932132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:31.932440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:31.933628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:31.933773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:31.938302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:31.938833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4d1bc68f-c024-4d93-bc06-23c5f0fa84c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:31.972502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 860/1715] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:31.973903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:32.005260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af065185-35d1-4c6c-9677-594f4121d7fe admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:32.023578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d8506-09e8-4c18-aedd-2d207d98968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:32.024368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d8506-09e8-4c18-aedd-2d207d98968a admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:32.030078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d8506-09e8-4c18-aedd-2d207d98968a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:32.030398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d8506-09e8-4c18-aedd-2d207d98968a admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:32.035559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d8506-09e8-4c18-aedd-2d207d98968a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:32.036592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d8506-09e8-4c18-aedd-2d207d98968a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:32.232953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 861/1716] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:31 2022] POST /identity/v3/projects => generated 371 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:32.234427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 856/1717] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:32.236696 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d8506-09e8-4c18-aedd-2d207d98968a admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:32.478263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d8506-09e8-4c18-aedd-2d207d98968a admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:32.486014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345d157-d91d-4ca3-8412-8e95615d72ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:32.487028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345d157-d91d-4ca3-8412-8e95615d72ef admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:32.487823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345d157-d91d-4ca3-8412-8e95615d72ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:32.487823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345d157-d91d-4ca3-8412-8e95615d72ef admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:32.488706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345d157-d91d-4ca3-8412-8e95615d72ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:32.490985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345d157-d91d-4ca3-8412-8e95615d72ef admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:32.592233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc68f-c024-4d93-bc06-23c5f0fa84c7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:32.594884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4d1bc68f-c024-4d93-bc06-23c5f0fa84c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:32.653042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc68f-c024-4d93-bc06-23c5f0fa84c7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:32.665861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 862/1718] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:32 2022] POST /identity/v3/users => generated 286 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:32.667233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345d157-d91d-4ca3-8412-8e95615d72ef admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:32.727966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345d157-d91d-4ca3-8412-8e95615d72ef admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:32.773097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44c74-8aa2-4d32-90fa-4b5310788ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:32.773097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44c74-8aa2-4d32-90fa-4b5310788ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:32.773097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44c74-8aa2-4d32-90fa-4b5310788ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:32.773097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c44c74-8aa2-4d32-90fa-4b5310788ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:32.773841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c44c74-8aa2-4d32-90fa-4b5310788ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:32.773841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c44c74-8aa2-4d32-90fa-4b5310788ad8 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:32.773841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 863/1719] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:32.777717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44c74-8aa2-4d32-90fa-4b5310788ad8 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:32.817587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c44c74-8aa2-4d32-90fa-4b5310788ad8 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:32.827325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8803a2-caaf-46fb-a2fd-7227608dc265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:32.831641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8803a2-caaf-46fb-a2fd-7227608dc265 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:32.831641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8803a2-caaf-46fb-a2fd-7227608dc265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:32.832910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8803a2-caaf-46fb-a2fd-7227608dc265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:32.833117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8803a2-caaf-46fb-a2fd-7227608dc265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:32.835695 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8803a2-caaf-46fb-a2fd-7227608dc265 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:32.852436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 864/1720] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:32.857132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8803a2-caaf-46fb-a2fd-7227608dc265 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:32.918707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8803a2-caaf-46fb-a2fd-7227608dc265 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:32.936016 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:32.942265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:32.942265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:32.942265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:32.954607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b02e57-df50-4608-8fa5-e5d8869d5954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:32.960145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b02e57-df50-4608-8fa5-e5d8869d5954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:32.961843 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b02e57-df50-4608-8fa5-e5d8869d5954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:32.964069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b02e57-df50-4608-8fa5-e5d8869d5954 admin admin] PATH_INFO: `/v3/projects/7bc42857e5da4f71ab6a491afa62aabf/users/c6de6fdea9704589af036fc1cd945022/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:32.997779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:32.999923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:33.019685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b02e57-df50-4608-8fa5-e5d8869d5954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc42857e5da4f71ab6a491afa62aabf, user_id=c6de6fdea9704589af036fc1cd945022, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:33.032252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 857/1721] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:33.034181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:33.042871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:33.046615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b02e57-df50-4608-8fa5-e5d8869d5954 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:33.084675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa055185-18ad-483a-ba19-bf18c1af6c7e service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:33.096816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:33.098993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:33.099540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:33.099842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] PATH_INFO: `/v3/users/dddb85c15f2d4606921b0818a296b296` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:33.111672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dddb85c15f2d4606921b0818a296b296)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:33.114064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:33.235125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 865/1722] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:32 2022] PUT /identity/v3/projects/7bc42857e5da4f71ab6a491afa62aabf/users/c6de6fdea9704589af036fc1cd945022/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:33.247864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875547d7-661f-4fcb-8921-2c9e06eef1c0 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:33.248214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875547d7-661f-4fcb-8921-2c9e06eef1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:33.248630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875547d7-661f-4fcb-8921-2c9e06eef1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:33.376344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-875547d7-661f-4fcb-8921-2c9e06eef1c0 None None] MFA Rules not processed for user `c6de6fdea9704589af036fc1cd945022`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:33.381157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-875547d7-661f-4fcb-8921-2c9e06eef1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:33.480545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dddb85c15f2d4606921b0818a296b296'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:33.509311 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dddb85c15f2d4606921b0818a296b296'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:33.528629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dddb85c15f2d4606921b0818a296b296'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:33.540876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dddb85c15f2d4606921b0818a296b296'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:33.600497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 866/1723] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:33.604740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875547d7-661f-4fcb-8921-2c9e06eef1c0 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:33.606647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-875547d7-661f-4fcb-8921-2c9e06eef1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:33.634184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9e9b9086-2d3d-45e9-aacc-deebf7abb24b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dddb85c15f2d4606921b0818a296b296'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:33.713899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 858/1724] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:33 2022] DELETE /identity/v3/users/dddb85c15f2d4606921b0818a296b296 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:33.717400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66377ba6-ed00-40fb-a5fd-80b5c9b96d57 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:33.717951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66377ba6-ed00-40fb-a5fd-80b5c9b96d57 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:33.718423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66377ba6-ed00-40fb-a5fd-80b5c9b96d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:33.765265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-66377ba6-ed00-40fb-a5fd-80b5c9b96d57 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:33.765265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-66377ba6-ed00-40fb-a5fd-80b5c9b96d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:33.936297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875547d7-661f-4fcb-8921-2c9e06eef1c0 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:33.965040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 859/1725] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:34.036987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66377ba6-ed00-40fb-a5fd-80b5c9b96d57 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:34.039077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-66377ba6-ed00-40fb-a5fd-80b5c9b96d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:34.106413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f02f95-6706-4d7f-9ef1-915b9aced80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:34.107139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f02f95-6706-4d7f-9ef1-915b9aced80f admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:34.108158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f02f95-6706-4d7f-9ef1-915b9aced80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:34.108455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f02f95-6706-4d7f-9ef1-915b9aced80f admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:34.109187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f02f95-6706-4d7f-9ef1-915b9aced80f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:34.137485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f02f95-6706-4d7f-9ef1-915b9aced80f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:34.277140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 867/1726] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:33 2022] POST /identity/v3/projects => generated 373 bytes in 673 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:34.300741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f02f95-6706-4d7f-9ef1-915b9aced80f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:34.326072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66377ba6-ed00-40fb-a5fd-80b5c9b96d57 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:34.509488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f02f95-6706-4d7f-9ef1-915b9aced80f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:34.515393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e805db-a4bb-4f33-9530-519d7f098bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:34.516113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e805db-a4bb-4f33-9530-519d7f098bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:34.516458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e805db-a4bb-4f33-9530-519d7f098bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:34.516743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e805db-a4bb-4f33-9530-519d7f098bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:34.517899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e805db-a4bb-4f33-9530-519d7f098bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:34.520931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e805db-a4bb-4f33-9530-519d7f098bd9 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:34.599634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf5668-6bca-4613-8983-7f1a36cf8cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:34.601695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf5668-6bca-4613-8983-7f1a36cf8cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:34.602221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf5668-6bca-4613-8983-7f1a36cf8cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:34.602739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf5668-6bca-4613-8983-7f1a36cf8cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:34.635055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 868/1727] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:34 2022] POST /identity/v3/users => generated 293 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:34.635148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf5668-6bca-4613-8983-7f1a36cf8cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:34.635643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf5668-6bca-4613-8983-7f1a36cf8cc7 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:34.644332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e805db-a4bb-4f33-9530-519d7f098bd9 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:34.655092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 860/1728] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:34.674253 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e805db-a4bb-4f33-9530-519d7f098bd9 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:34.690572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95bea1-619c-460d-88c2-aede522c8f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:34.691286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95bea1-619c-460d-88c2-aede522c8f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:34.691575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95bea1-619c-460d-88c2-aede522c8f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:34.691892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95bea1-619c-460d-88c2-aede522c8f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:34.692756 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95bea1-619c-460d-88c2-aede522c8f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:34.695579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95bea1-619c-460d-88c2-aede522c8f90 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:34.731823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 869/1729] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:34.735543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf5668-6bca-4613-8983-7f1a36cf8cc7 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:34.802658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf5668-6bca-4613-8983-7f1a36cf8cc7 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:34.802658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dfe42-90e0-4984-8c82-8c9b41173b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:34.802658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dfe42-90e0-4984-8c82-8c9b41173b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:34.802658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dfe42-90e0-4984-8c82-8c9b41173b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:34.803170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dfe42-90e0-4984-8c82-8c9b41173b47 admin admin] PATH_INFO: `/v3/projects/b52ff33ad8e54f8d8fc882f2a0862b95/users/eb77c07988134b6f83528b4d9ac248a8/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:34.803170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dfe42-90e0-4984-8c82-8c9b41173b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52ff33ad8e54f8d8fc882f2a0862b95, user_id=eb77c07988134b6f83528b4d9ac248a8, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:34.803170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dfe42-90e0-4984-8c82-8c9b41173b47 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:34.976824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 861/1730] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:34 2022] PUT /identity/v3/projects/b52ff33ad8e54f8d8fc882f2a0862b95/users/eb77c07988134b6f83528b4d9ac248a8/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:34.986480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95bea1-619c-460d-88c2-aede522c8f90 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:34.997443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95bea1-619c-460d-88c2-aede522c8f90 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:35.148093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9eb6f-16ad-4fa0-975f-4f25ea4f6951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:35.148854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9eb6f-16ad-4fa0-975f-4f25ea4f6951 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:35.149143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9eb6f-16ad-4fa0-975f-4f25ea4f6951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:35.149417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9eb6f-16ad-4fa0-975f-4f25ea4f6951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:35.153444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9eb6f-16ad-4fa0-975f-4f25ea4f6951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:35.155426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9eb6f-16ad-4fa0-975f-4f25ea4f6951 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:35.193060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dfe42-90e0-4984-8c82-8c9b41173b47 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:35.205262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dfe42-90e0-4984-8c82-8c9b41173b47 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:35.207311 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 870/1731] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:35.216385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:35.217817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:35.218359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:35.218863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] PATH_INFO: `/v3/projects/b52ff33ad8e54f8d8fc882f2a0862b95/users/eb77c07988134b6f83528b4d9ac248a8/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:35.221639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52ff33ad8e54f8d8fc882f2a0862b95, user_id=eb77c07988134b6f83528b4d9ac248a8, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:35.233641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:35.257296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 862/1732] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:35 2022] PUT /identity/v3/projects/b52ff33ad8e54f8d8fc882f2a0862b95/users/eb77c07988134b6f83528b4d9ac248a8/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:35.268720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeea1b-fd80-4c51-9f0b-af1c9bb79ee9 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:35.269243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeea1b-fd80-4c51-9f0b-af1c9bb79ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:35.269664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeea1b-fd80-4c51-9f0b-af1c9bb79ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:35.333061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-5efeea1b-fd80-4c51-9f0b-af1c9bb79ee9 None None] MFA Rules not processed for user `eb77c07988134b6f83528b4d9ac248a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:35.334777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5efeea1b-fd80-4c51-9f0b-af1c9bb79ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:35.517714 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 871/1733] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:35.530526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:35.532777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:35.548692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeea1b-fd80-4c51-9f0b-af1c9bb79ee9 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:35.552199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5efeea1b-fd80-4c51-9f0b-af1c9bb79ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:35.663576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f1b31-95b8-42df-afc5-b0c299ab9d50 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:35.691507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989eeb8-ac95-4d7b-8308-8cce6539fdcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:35.691892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeea1b-fd80-4c51-9f0b-af1c9bb79ee9 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:35.694696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989eeb8-ac95-4d7b-8308-8cce6539fdcc service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:35.694696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989eeb8-ac95-4d7b-8308-8cce6539fdcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:35.694696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989eeb8-ac95-4d7b-8308-8cce6539fdcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:35.702802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:35.705254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:35.705254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:35.706226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] PATH_INFO: `/v3/projects/e31f5d197d024572b044da9343a9b4a1` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:35.708441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31f5d197d024572b044da9343a9b4a1)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:35.711805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:35.733573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989eeb8-ac95-4d7b-8308-8cce6539fdcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:35.737957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989eeb8-ac95-4d7b-8308-8cce6539fdcc service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:35.758562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 863/1734] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:35.763237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec51f-01a9-4461-8113-e1d51a0cd3ba None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:35.763554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec51f-01a9-4461-8113-e1d51a0cd3ba None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:35.779950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec51f-01a9-4461-8113-e1d51a0cd3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:35.818816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d93ec51f-01a9-4461-8113-e1d51a0cd3ba None None] MFA Rules not processed for user `233f5342cf334c7689cb9e71f4fbaab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:35.821005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d93ec51f-01a9-4461-8113-e1d51a0cd3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:35.999474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31f5d197d024572b044da9343a9b4a1', 'request_id': 'req-50b4f320-62f6-476d-b022-ad734d75e834'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.003870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 864/1735] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:36.014709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31f5d197d024572b044da9343a9b4a1', 'request_id': 'req-50b4f320-62f6-476d-b022-ad734d75e834'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.202418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31f5d197d024572b044da9343a9b4a1', 'request_id': 'req-50b4f320-62f6-476d-b022-ad734d75e834'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.231674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] The token cache is being invalidate because project e31f5d197d024572b044da9343a9b4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:36.231977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.232689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.239038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec51f-01a9-4461-8113-e1d51a0cd3ba None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:36.240604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 872/1736] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:35 2022] DELETE /identity/v3/projects/e31f5d197d024572b044da9343a9b4a1 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:36.261728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec51f-01a9-4461-8113-e1d51a0cd3ba None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:36.379256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:36.380490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:36.381052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:36.381391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] PATH_INFO: `/v3/users/181bf5b37b81456e908f699c07240175` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:36.383159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=181bf5b37b81456e908f699c07240175)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:36.385652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:36.458074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:36.460440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:36.504825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4f320-62f6-476d-b022-ad734d75e834 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:36.520633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:36.522072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:36.525941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:36.525941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] PATH_INFO: `/v3/projects/b6d5e50de93044abbe8157e2085f63a6` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:36.525941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d5e50de93044abbe8157e2085f63a6)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:36.528766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:36.630165 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181bf5b37b81456e908f699c07240175'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.632471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181bf5b37b81456e908f699c07240175'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.634233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181bf5b37b81456e908f699c07240175'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.650273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181bf5b37b81456e908f699c07240175'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.751857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181bf5b37b81456e908f699c07240175'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.771021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d5e50de93044abbe8157e2085f63a6', 'request_id': 'req-800a82c0-f759-4ba8-b98e-4a52a32ba04d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.835924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d5e50de93044abbe8157e2085f63a6', 'request_id': 'req-800a82c0-f759-4ba8-b98e-4a52a32ba04d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.889818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 865/1737] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:36 2022] DELETE /identity/v3/users/181bf5b37b81456e908f699c07240175 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:36.964699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d5e50de93044abbe8157e2085f63a6', 'request_id': 'req-800a82c0-f759-4ba8-b98e-4a52a32ba04d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:36.992928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] The token cache is being invalidate because project b6d5e50de93044abbe8157e2085f63a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:36.995858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:37.003627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:37.007734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 873/1738] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:36 2022] DELETE /identity/v3/projects/b6d5e50de93044abbe8157e2085f63a6 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:37.013237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:37.014900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:37.365930 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb9f4a-b6a3-441b-a37d-d11cc23af406 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:37.379209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:37.380360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:37.380849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:37.381319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] PATH_INFO: `/v3/users/5dee00f7ded244dcaadfb479e18bb25a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:37.382848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dee00f7ded244dcaadfb479e18bb25a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:37.385185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:37.502360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:37.509185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:37.628542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a82c0-f759-4ba8-b98e-4a52a32ba04d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:37.643119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:37.644603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:37.644603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:37.644603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] PATH_INFO: `/v3/users/c6de6fdea9704589af036fc1cd945022` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:37.648404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dee00f7ded244dcaadfb479e18bb25a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:37.654985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dee00f7ded244dcaadfb479e18bb25a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:37.656144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dee00f7ded244dcaadfb479e18bb25a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:37.666499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dee00f7ded244dcaadfb479e18bb25a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:37.667467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6de6fdea9704589af036fc1cd945022)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:37.670226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:37.770904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dee00f7ded244dcaadfb479e18bb25a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:37.839474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 866/1739] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:37 2022] DELETE /identity/v3/users/5dee00f7ded244dcaadfb479e18bb25a => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:37.986613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6de6fdea9704589af036fc1cd945022'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:37.988548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6de6fdea9704589af036fc1cd945022'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:38.184291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6de6fdea9704589af036fc1cd945022'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:38.199721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:38.214971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:38.235107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6de6fdea9704589af036fc1cd945022'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:38.238513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-64f251ab-6c31-4c0a-a9c4-b6e4bdc09a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6de6fdea9704589af036fc1cd945022'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:38.370568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 874/1740] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:37 2022] DELETE /identity/v3/users/c6de6fdea9704589af036fc1cd945022 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:38.373317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca80fe-488c-480b-af5f-e23cf783cd33 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:38.376775 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca80fe-488c-480b-af5f-e23cf783cd33 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:38.377372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca80fe-488c-480b-af5f-e23cf783cd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:38.439524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bbca80fe-488c-480b-af5f-e23cf783cd33 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:38.443149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bbca80fe-488c-480b-af5f-e23cf783cd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:38.502601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a72ee-e83a-4e65-9224-be4838d7ad8a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:38.643096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 875/1741] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:38.677966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:38.678516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:38.678799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:38.679262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] PATH_INFO: `/v3/projects/03c2fc33ceb84ef5b6a4c5b6027c91b4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:38.681561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03c2fc33ceb84ef5b6a4c5b6027c91b4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:38.686525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:38.692944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca80fe-488c-480b-af5f-e23cf783cd33 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:38.694889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bbca80fe-488c-480b-af5f-e23cf783cd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:39.137875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03c2fc33ceb84ef5b6a4c5b6027c91b4', 'request_id': 'req-99c05b95-91c4-4014-b33c-37b0be563137'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:39.138443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca80fe-488c-480b-af5f-e23cf783cd33 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:39.138529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03c2fc33ceb84ef5b6a4c5b6027c91b4', 'request_id': 'req-99c05b95-91c4-4014-b33c-37b0be563137'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:39.210112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:39.210112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:39.210112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:39.210112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:39.391219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:39.391219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:39.391219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 876/1742] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:39.737013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03c2fc33ceb84ef5b6a4c5b6027c91b4', 'request_id': 'req-99c05b95-91c4-4014-b33c-37b0be563137'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:39.747357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:39.749869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:39.772839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] The token cache is being invalidate because project 03c2fc33ceb84ef5b6a4c5b6027c91b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:39.773351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:39.774054 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-99c05b95-91c4-4014-b33c-37b0be563137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:39.777221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2a618-e7f2-4f4f-abeb-b0f935d153fa service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:39.781102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 867/1743] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:38 2022] DELETE /identity/v3/projects/03c2fc33ceb84ef5b6a4c5b6027c91b4 => generated 0 bytes in 1582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:39.787258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa47bb-53d7-4e19-a8fc-5a1f3dc9709c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:39.787258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa47bb-53d7-4e19-a8fc-5a1f3dc9709c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:39.787809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa47bb-53d7-4e19-a8fc-5a1f3dc9709c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:39.792607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:39.794569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:39.795365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:39.795671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] PATH_INFO: `/v3/projects/6b022327bc00410aa8a191c73fbacf8c` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:39.799901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b022327bc00410aa8a191c73fbacf8c)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:39.805685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:39.819858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bffa47bb-53d7-4e19-a8fc-5a1f3dc9709c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:39.821368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bffa47bb-53d7-4e19-a8fc-5a1f3dc9709c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:39.831083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 868/1744] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:39.839251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7125c4-d3bb-41df-a51b-ee6b2e438e6d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:39.839723 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7125c4-d3bb-41df-a51b-ee6b2e438e6d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:39.840551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7125c4-d3bb-41df-a51b-ee6b2e438e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:39.869344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3e7125c4-d3bb-41df-a51b-ee6b2e438e6d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:39.870727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3e7125c4-d3bb-41df-a51b-ee6b2e438e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:39.881047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 869/1745] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:39.885058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7125c4-d3bb-41df-a51b-ee6b2e438e6d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:39.897992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7125c4-d3bb-41df-a51b-ee6b2e438e6d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:39.903757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:39.904531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:39.904826 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:39.905104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:39.905981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:39.909341 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:39.909341 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:39.934130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 870/1746] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:25:39.942127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:39.956977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68468710-e54d-443d-9208-a7d6cc47340c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:39.966809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b0e8-6173-4749-8a32-b688d55fa1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:39.967783 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b0e8-6173-4749-8a32-b688d55fa1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:39.968416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b0e8-6173-4749-8a32-b688d55fa1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:39.968877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b0e8-6173-4749-8a32-b688d55fa1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:39.969810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1b0e8-6173-4749-8a32-b688d55fa1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:39.972104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1b0e8-6173-4749-8a32-b688d55fa1c0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:39.998489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b022327bc00410aa8a191c73fbacf8c', 'request_id': 'req-e076136e-3197-4a03-b881-89476ffef31a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.006604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b022327bc00410aa8a191c73fbacf8c', 'request_id': 'req-e076136e-3197-4a03-b881-89476ffef31a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.121198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b022327bc00410aa8a191c73fbacf8c', 'request_id': 'req-e076136e-3197-4a03-b881-89476ffef31a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.129293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 871/1747] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:39 2022] POST /identity/v3/projects => generated 365 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:40.133255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b0e8-6173-4749-8a32-b688d55fa1c0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:40.140018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] The token cache is being invalidate because project 6b022327bc00410aa8a191c73fbacf8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:40.140639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.141490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.143908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 877/1748] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:39 2022] DELETE /identity/v3/projects/6b022327bc00410aa8a191c73fbacf8c => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:40.149491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:40.151698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:40.265466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b0e8-6173-4749-8a32-b688d55fa1c0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:40.275228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:40.275574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076136e-3197-4a03-b881-89476ffef31a admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:40.276469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:40.277606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:40.278154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:40.281340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:40.285393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:40.288380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:40.290711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:40.293181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:40.293685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] PATH_INFO: `/v3/users/3f8d7466e3724f38b4a86d7720726338` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:40.297485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f8d7466e3724f38b4a86d7720726338)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:40.304773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:40.442134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 872/1749] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:40 2022] POST /identity/v3/users => generated 283 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:40.449420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:40.451988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:40.478382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7d6c5-b7c1-4275-bd33-2b187d439101 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:40.488838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:40.489766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:40.492179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:40.492573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:40.493460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:40.507797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:40.551792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f8d7466e3724f38b4a86d7720726338'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.552722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f8d7466e3724f38b4a86d7720726338'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.566740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 873/1750] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:40.574924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:40.577037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:40.593083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62887-3f2c-494d-a3aa-e6a14f543bb7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:40.599185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b022-056d-46b8-89ab-59376ff29d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:40.600092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b022-056d-46b8-89ab-59376ff29d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:40.600542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b022-056d-46b8-89ab-59376ff29d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:40.600976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b022-056d-46b8-89ab-59376ff29d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:40.601921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092b022-056d-46b8-89ab-59376ff29d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:40.603722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092b022-056d-46b8-89ab-59376ff29d90 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:40.625024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 874/1751] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:40.627160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b022-056d-46b8-89ab-59376ff29d90 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:40.640403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b022-056d-46b8-89ab-59376ff29d90 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:40.650573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:40.651511 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:40.651788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:40.652070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] PATH_INFO: `/v3/projects/5590fd98524b44eb8d71dc59fefa6993/users/bb70755b1a4d441fbe11fc2565bc7ae8/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:40.669216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5590fd98524b44eb8d71dc59fefa6993, user_id=bb70755b1a4d441fbe11fc2565bc7ae8, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:40.673521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f8d7466e3724f38b4a86d7720726338'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.676174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:40.688191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f8d7466e3724f38b4a86d7720726338'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.688191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0f4dfc31-b8e5-4e5b-bf65-be0316029e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f8d7466e3724f38b4a86d7720726338'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:40.719022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 878/1752] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:40 2022] DELETE /identity/v3/users/3f8d7466e3724f38b4a86d7720726338 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:40.787390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 875/1753] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:40 2022] PUT /identity/v3/projects/5590fd98524b44eb8d71dc59fefa6993/users/bb70755b1a4d441fbe11fc2565bc7ae8/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:40.800289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b471d-c3cc-49aa-a8f0-1382ac227ceb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:40.801387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b471d-c3cc-49aa-a8f0-1382ac227ceb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:40.801854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b471d-c3cc-49aa-a8f0-1382ac227ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:40.841652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-272b471d-c3cc-49aa-a8f0-1382ac227ceb None None] MFA Rules not processed for user `bb70755b1a4d441fbe11fc2565bc7ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:40.844326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-272b471d-c3cc-49aa-a8f0-1382ac227ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:40.892645 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:40.894614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:40.981094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 879/1754] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:40.986745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b471d-c3cc-49aa-a8f0-1382ac227ceb None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:41.031914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad00d3-f015-4e9f-80f3-6761b32b455f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:41.037871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:41.039547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:41.040202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:41.040569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] PATH_INFO: `/v3/projects/7bc42857e5da4f71ab6a491afa62aabf` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:41.042073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc42857e5da4f71ab6a491afa62aabf)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:41.044275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:41.142335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b471d-c3cc-49aa-a8f0-1382ac227ceb None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:41.147054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c276d7-137b-4711-b0ef-49d2710ab02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:41.151423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c276d7-137b-4711-b0ef-49d2710ab02f admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:41.151423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c276d7-137b-4711-b0ef-49d2710ab02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:41.151423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c276d7-137b-4711-b0ef-49d2710ab02f admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:41.151423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c276d7-137b-4711-b0ef-49d2710ab02f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:41.151423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c276d7-137b-4711-b0ef-49d2710ab02f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:41.187810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc42857e5da4f71ab6a491afa62aabf', 'request_id': 'req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.192950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc42857e5da4f71ab6a491afa62aabf', 'request_id': 'req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.264748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 880/1755] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:40 2022] POST /identity/v3/projects => generated 363 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:41.268976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c276d7-137b-4711-b0ef-49d2710ab02f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:41.280036 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc42857e5da4f71ab6a491afa62aabf', 'request_id': 'req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.303900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] The token cache is being invalidate because project 7bc42857e5da4f71ab6a491afa62aabf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:41.304255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.307275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.313680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 876/1756] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:40 2022] DELETE /identity/v3/projects/7bc42857e5da4f71ab6a491afa62aabf => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:41.315058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:41.323513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:41.350915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c276d7-137b-4711-b0ef-49d2710ab02f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:41.368071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:41.369308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:41.369750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:41.370198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:41.371392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:41.376841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:41.416755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb9315-8e00-4e95-b1ea-9d52dabdb27a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:41.421714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:41.422816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:41.423176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:41.423514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] PATH_INFO: `/v3/users/eb77c07988134b6f83528b4d9ac248a8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:41.424995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb77c07988134b6f83528b4d9ac248a8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:41.427083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:41.481413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 881/1757] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:41 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:41.486331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:41.489613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:41.528257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f072c-be25-4846-bf83-7c7bb0a90685 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:41.559662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa5c56-da31-41a0-a992-b3e7399d9db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:41.574032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5c56-da31-41a0-a992-b3e7399d9db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:41.577686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5c56-da31-41a0-a992-b3e7399d9db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:41.580348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5c56-da31-41a0-a992-b3e7399d9db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:41.581310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa5c56-da31-41a0-a992-b3e7399d9db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:41.583147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa5c56-da31-41a0-a992-b3e7399d9db7 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:41.619709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 882/1758] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:41.622139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa5c56-da31-41a0-a992-b3e7399d9db7 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:41.635081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa5c56-da31-41a0-a992-b3e7399d9db7 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:41.643985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:41.646787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:41.646787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:41.646787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] PATH_INFO: `/v3/projects/89cb9b363e4a4d6b913dea40bdbe07b6/users/b848f524d8a14915965284767139a4a2/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:41.664581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb77c07988134b6f83528b4d9ac248a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.667041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb77c07988134b6f83528b4d9ac248a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.667697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb77c07988134b6f83528b4d9ac248a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.668806 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cb9b363e4a4d6b913dea40bdbe07b6, user_id=b848f524d8a14915965284767139a4a2, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:41.672634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:41.687010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb77c07988134b6f83528b4d9ac248a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.774023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 883/1759] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:41 2022] PUT /identity/v3/projects/89cb9b363e4a4d6b913dea40bdbe07b6/users/b848f524d8a14915965284767139a4a2/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:41.775366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:41.780259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:41.794883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb77c07988134b6f83528b4d9ac248a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:41.827398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 877/1760] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:41 2022] DELETE /identity/v3/users/eb77c07988134b6f83528b4d9ac248a8 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:41.887520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5963abb-2140-4160-a72f-cff321bd293c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:41.956616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ee59d-fef9-4eef-b1e5-10eee3b86e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:41.957830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ee59d-fef9-4eef-b1e5-10eee3b86e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:41.958103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ee59d-fef9-4eef-b1e5-10eee3b86e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:41.958445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ee59d-fef9-4eef-b1e5-10eee3b86e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:41.959275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ee59d-fef9-4eef-b1e5-10eee3b86e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:41.962417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ee59d-fef9-4eef-b1e5-10eee3b86e85 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:41.975390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 884/1761] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:41.979649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:42.007051 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f85924-55f7-4482-b4cf-d0a79552f9f0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:42.017566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:42.019723 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:42.020108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:42.020519 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] PATH_INFO: `/v3/projects/89cb9b363e4a4d6b913dea40bdbe07b6/users/b848f524d8a14915965284767139a4a2/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:42.023452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cb9b363e4a4d6b913dea40bdbe07b6, user_id=b848f524d8a14915965284767139a4a2, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:42.026829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:42.042263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 878/1762] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:41 2022] PUT /identity/v3/projects/89cb9b363e4a4d6b913dea40bdbe07b6/users/b848f524d8a14915965284767139a4a2/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:42.056381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1fa6a-2042-4a89-b481-2768a7dc804a None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:42.056728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1fa6a-2042-4a89-b481-2768a7dc804a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:42.057700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1fa6a-2042-4a89-b481-2768a7dc804a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:42.108329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f9e1fa6a-2042-4a89-b481-2768a7dc804a None None] MFA Rules not processed for user `b848f524d8a14915965284767139a4a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:42.111981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f9e1fa6a-2042-4a89-b481-2768a7dc804a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:42.213489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:42.215485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:42.226276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 885/1763] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:42 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:42.237511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1fa6a-2042-4a89-b481-2768a7dc804a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:42.239239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f9e1fa6a-2042-4a89-b481-2768a7dc804a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:42.341819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181e99-b551-4cd3-a7fa-833f70c2bc22 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:42.346444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1fa6a-2042-4a89-b481-2768a7dc804a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:42.348543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:42.349723 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:42.350203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:42.350740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] PATH_INFO: `/v3/projects/cf314aaa1bb045edb828a31c17f28777` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:42.351993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf314aaa1bb045edb828a31c17f28777)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:42.352126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6bd87-bbb1-43e9-9480-bde5680b640a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:42.353335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6bd87-bbb1-43e9-9480-bde5680b640a service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:42.353703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6bd87-bbb1-43e9-9480-bde5680b640a service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:42.354148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6bd87-bbb1-43e9-9480-bde5680b640a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:42.354551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:42.370638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6bd87-bbb1-43e9-9480-bde5680b640a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:42.373527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6bd87-bbb1-43e9-9480-bde5680b640a service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:42.429500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 886/1764] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:42 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:42.582729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf314aaa1bb045edb828a31c17f28777', 'request_id': 'req-68927cf1-d454-4a7b-b39b-c143dc8b77a7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:42.597379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf314aaa1bb045edb828a31c17f28777', 'request_id': 'req-68927cf1-d454-4a7b-b39b-c143dc8b77a7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:42.782249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf314aaa1bb045edb828a31c17f28777', 'request_id': 'req-68927cf1-d454-4a7b-b39b-c143dc8b77a7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:42.799042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] The token cache is being invalidate because project cf314aaa1bb045edb828a31c17f28777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:42.799542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:42.800557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:42.802837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 879/1765] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:42 2022] DELETE /identity/v3/projects/cf314aaa1bb045edb828a31c17f28777 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:42.810024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872689c6-35ef-4645-b469-2bb001f28405 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:42.811289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872689c6-35ef-4645-b469-2bb001f28405 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:42.811763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872689c6-35ef-4645-b469-2bb001f28405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:42.839966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-872689c6-35ef-4645-b469-2bb001f28405 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:42.843754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-872689c6-35ef-4645-b469-2bb001f28405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:42.899170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:42.906952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:42.946467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 887/1766] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:42.955208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d713-40ae-4c3f-81b7-0165076b772d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:42.955795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d713-40ae-4c3f-81b7-0165076b772d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:42.956316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d713-40ae-4c3f-81b7-0165076b772d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:42.975846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-9b19d713-40ae-4c3f-81b7-0165076b772d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:42.977278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9b19d713-40ae-4c3f-81b7-0165076b772d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:42.983710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68927cf1-d454-4a7b-b39b-c143dc8b77a7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:42.986810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 888/1767] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:42.990977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19d713-40ae-4c3f-81b7-0165076b772d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:42.991297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39617ddd-cfa9-4342-80a1-0024d3b6944a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:42.991962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39617ddd-cfa9-4342-80a1-0024d3b6944a service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:42.992896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39617ddd-cfa9-4342-80a1-0024d3b6944a service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:42.993179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39617ddd-cfa9-4342-80a1-0024d3b6944a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:42.995580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-39617ddd-cfa9-4342-80a1-0024d3b6944a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:43.000431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19d713-40ae-4c3f-81b7-0165076b772d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:43.005967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:43.006703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.007204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.007544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.008351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:43.008667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:43.010499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:43.025004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 889/1768] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:43.030479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:43.042397 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b280c1-2c48-42fd-bd37-6990d6ac9c77 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:43.047179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1dae-e8d4-4411-a7a9-f80a12d7f639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:43.047958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1dae-e8d4-4411-a7a9-f80a12d7f639 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.048314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1dae-e8d4-4411-a7a9-f80a12d7f639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.048795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1dae-e8d4-4411-a7a9-f80a12d7f639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.051255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1dae-e8d4-4411-a7a9-f80a12d7f639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:43.070558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1dae-e8d4-4411-a7a9-f80a12d7f639 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:43.071373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39617ddd-cfa9-4342-80a1-0024d3b6944a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:43.072972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39617ddd-cfa9-4342-80a1-0024d3b6944a service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:43.094596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 880/1769] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:43.249283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed43a3-8a98-4d7d-acd2-0f9f83f68db4 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.254706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed43a3-8a98-4d7d-acd2-0f9f83f68db4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.254706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed43a3-8a98-4d7d-acd2-0f9f83f68db4 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.258453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 890/1770] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:43 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:43.259481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 881/1771] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:25:43 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:25:43.263330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1dae-e8d4-4411-a7a9-f80a12d7f639 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:43.272502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec409-d1fd-40c7-837e-819dc70fd0e1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.272899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec409-d1fd-40c7-837e-819dc70fd0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.273358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec409-d1fd-40c7-837e-819dc70fd0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.399793 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-598ec409-d1fd-40c7-837e-819dc70fd0e1 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:43.407197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-598ec409-d1fd-40c7-837e-819dc70fd0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:43.530298 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1dae-e8d4-4411-a7a9-f80a12d7f639 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:43.536044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b93c4-e253-4660-924b-fe070d326b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:43.539188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b93c4-e253-4660-924b-fe070d326b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.539188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b93c4-e253-4660-924b-fe070d326b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.539188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b93c4-e253-4660-924b-fe070d326b77 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.539188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b93c4-e253-4660-924b-fe070d326b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:43.541660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b93c4-e253-4660-924b-fe070d326b77 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:43.614216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 882/1772] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:25:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:43.646258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 891/1773] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:43 2022] POST /identity/v3/users => generated 294 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:43.650737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec409-d1fd-40c7-837e-819dc70fd0e1 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:43.670003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec409-d1fd-40c7-837e-819dc70fd0e1 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:43.685687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e371a-ee2c-4bb5-a362-5acf4c369ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:43.687985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e371a-ee2c-4bb5-a362-5acf4c369ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.689479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e371a-ee2c-4bb5-a362-5acf4c369ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.689888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e371a-ee2c-4bb5-a362-5acf4c369ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.695569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e371a-ee2c-4bb5-a362-5acf4c369ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:43.697261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e371a-ee2c-4bb5-a362-5acf4c369ce1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:43.726486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 883/1774] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:43.730753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b93c4-e253-4660-924b-fe070d326b77 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:43.745770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b93c4-e253-4660-924b-fe070d326b77 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:43.759852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91721d-63b5-4dac-87c3-9dd2c661bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:43.760551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91721d-63b5-4dac-87c3-9dd2c661bbce admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.760837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91721d-63b5-4dac-87c3-9dd2c661bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.761113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91721d-63b5-4dac-87c3-9dd2c661bbce admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.762388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91721d-63b5-4dac-87c3-9dd2c661bbce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:43.764061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91721d-63b5-4dac-87c3-9dd2c661bbce admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:43.776746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 892/1775] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:43.781424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e371a-ee2c-4bb5-a362-5acf4c369ce1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:43.794496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e371a-ee2c-4bb5-a362-5acf4c369ce1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:43.803271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:43.804499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.805224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.805632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] PATH_INFO: `/v3/projects/1331e21bd7ce4ffba3566d44c8342f7f/users/5cd0887121e7424ebd9440df1777f7e3/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.825241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1331e21bd7ce4ffba3566d44c8342f7f, user_id=5cd0887121e7424ebd9440df1777f7e3, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:43.828882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:43.951805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 884/1776] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:43 2022] PUT /identity/v3/projects/1331e21bd7ce4ffba3566d44c8342f7f/users/5cd0887121e7424ebd9440df1777f7e3/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:43.963921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ee60-9705-4db8-a49f-faf12298cc2b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:43.964548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ee60-9705-4db8-a49f-faf12298cc2b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:43.964548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ee60-9705-4db8-a49f-faf12298cc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:43.989409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:43.991202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:43.999121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-cbe5ee60-9705-4db8-a49f-faf12298cc2b None None] MFA Rules not processed for user `5cd0887121e7424ebd9440df1777f7e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:44.001385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-cbe5ee60-9705-4db8-a49f-faf12298cc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:44.127191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530008eb-4954-4314-8e85-2f41fe411952 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:44.139585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:44.140523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:44.140788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:44.141056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] PATH_INFO: `/v3/projects/b52ff33ad8e54f8d8fc882f2a0862b95` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:44.142709 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52ff33ad8e54f8d8fc882f2a0862b95)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:44.147382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 893/1777] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:44.147529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:44.147604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ee60-9705-4db8-a49f-faf12298cc2b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:44.148860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-cbe5ee60-9705-4db8-a49f-faf12298cc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:44.248454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ee60-9705-4db8-a49f-faf12298cc2b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:44.256190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31620-92b6-43fd-a701-f8fd0ca74efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:44.256857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31620-92b6-43fd-a701-f8fd0ca74efd service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:44.257626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31620-92b6-43fd-a701-f8fd0ca74efd service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:44.257942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31620-92b6-43fd-a701-f8fd0ca74efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:44.274386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc31620-92b6-43fd-a701-f8fd0ca74efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:44.276710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc31620-92b6-43fd-a701-f8fd0ca74efd service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:44.290087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 894/1778] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:44.291000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31620-92b6-43fd-a701-f8fd0ca74efd service nova] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:44.304506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31620-92b6-43fd-a701-f8fd0ca74efd service nova] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:44.316635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7734bb1-bb84-4fe1-89cc-e5bb389fe294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:44.317990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734bb1-bb84-4fe1-89cc-e5bb389fe294 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:44.318287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734bb1-bb84-4fe1-89cc-e5bb389fe294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:44.319199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734bb1-bb84-4fe1-89cc-e5bb389fe294 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:44.319912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7734bb1-bb84-4fe1-89cc-e5bb389fe294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:44.323121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7734bb1-bb84-4fe1-89cc-e5bb389fe294 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:44.361219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52ff33ad8e54f8d8fc882f2a0862b95', 'request_id': 'req-605ade10-89fc-4824-b879-82d3d939b0f6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:44.372553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52ff33ad8e54f8d8fc882f2a0862b95', 'request_id': 'req-605ade10-89fc-4824-b879-82d3d939b0f6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:45.131481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 895/1779] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:44 2022] POST /identity/v3/projects => generated 385 bytes in 841 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:45.135060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7734bb1-bb84-4fe1-89cc-e5bb389fe294 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:45.210157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52ff33ad8e54f8d8fc882f2a0862b95', 'request_id': 'req-605ade10-89fc-4824-b879-82d3d939b0f6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:45.228880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] The token cache is being invalidate because project b52ff33ad8e54f8d8fc882f2a0862b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:45.230465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:45.233090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-605ade10-89fc-4824-b879-82d3d939b0f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:45.246962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7734bb1-bb84-4fe1-89cc-e5bb389fe294 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:45.253310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:45.254299 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:45.289699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 885/1780] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:43 2022] DELETE /identity/v3/projects/b52ff33ad8e54f8d8fc882f2a0862b95 => generated 0 bytes in 1266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:45.290797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:45.290797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:45.290797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:45.290797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:45.294030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207967-6613-41b8-a3fb-6ad56d68dde0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:45.294191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207967-6613-41b8-a3fb-6ad56d68dde0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:45.295931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3207967-6613-41b8-a3fb-6ad56d68dde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:45.329657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c3207967-6613-41b8-a3fb-6ad56d68dde0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:45.331464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c3207967-6613-41b8-a3fb-6ad56d68dde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:45.341824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 886/1781] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:45.349762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a95448-82e8-48b9-a890-39c121056145 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:45.357559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a95448-82e8-48b9-a890-39c121056145 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:45.357956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a95448-82e8-48b9-a890-39c121056145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:45.382484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d0a95448-82e8-48b9-a890-39c121056145 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:45.383727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d0a95448-82e8-48b9-a890-39c121056145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:45.417043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 887/1782] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:45.420820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a95448-82e8-48b9-a890-39c121056145 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:45.468795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a95448-82e8-48b9-a890-39c121056145 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:45.559542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:45.559542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:45.559542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:45.559542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:45.561346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:45.561346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:45.561346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:45.561346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 888/1783] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:45.570842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:45.609987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a361a7-d75a-446c-89a0-4a7fe8ad26ac admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:45.631334 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4e4c8-fae7-42a5-8f50-e607daa4f5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:45.632233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4e4c8-fae7-42a5-8f50-e607daa4f5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:45.632518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4e4c8-fae7-42a5-8f50-e607daa4f5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:45.632796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4e4c8-fae7-42a5-8f50-e607daa4f5bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:45.633692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4e4c8-fae7-42a5-8f50-e607daa4f5bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:45.639064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4e4c8-fae7-42a5-8f50-e607daa4f5bd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.109104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 896/1784] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:45 2022] POST /identity/v3/users => generated 299 bytes in 974 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:46.115470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.117061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:46.129323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5a187-8d1d-4225-93aa-c775438c6db1 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.133835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49a209-ba7d-43f9-9c7c-e6c9c77890cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.134492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a209-ba7d-43f9-9c7c-e6c9c77890cc admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.134775 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a209-ba7d-43f9-9c7c-e6c9c77890cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.135049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a209-ba7d-43f9-9c7c-e6c9c77890cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.135803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49a209-ba7d-43f9-9c7c-e6c9c77890cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:46.137435 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49a209-ba7d-43f9-9c7c-e6c9c77890cc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.147405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 897/1785] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:46.151305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49a209-ba7d-43f9-9c7c-e6c9c77890cc admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.160753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49a209-ba7d-43f9-9c7c-e6c9c77890cc admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.165870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875591ff-0b2f-4307-9081-a152a0d3d73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.166754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875591ff-0b2f-4307-9081-a152a0d3d73d admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.167050 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875591ff-0b2f-4307-9081-a152a0d3d73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.167369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875591ff-0b2f-4307-9081-a152a0d3d73d admin admin] PATH_INFO: `/v3/projects/c29d54e57cc74b8b826640ba8da3c95d/users/65b2e405c9334dbda6e889f3025b5912/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.173362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 889/1786] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:45 2022] POST /identity/v3/projects => generated 367 bytes in 603 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:46.177487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4e4c8-fae7-42a5-8f50-e607daa4f5bd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.183289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875591ff-0b2f-4307-9081-a152a0d3d73d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29d54e57cc74b8b826640ba8da3c95d, user_id=65b2e405c9334dbda6e889f3025b5912, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:46.186626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875591ff-0b2f-4307-9081-a152a0d3d73d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.287387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 898/1787] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:46 2022] PUT /identity/v3/projects/c29d54e57cc74b8b826640ba8da3c95d/users/65b2e405c9334dbda6e889f3025b5912/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:46.290211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4e4c8-fae7-42a5-8f50-e607daa4f5bd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.294120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875591ff-0b2f-4307-9081-a152a0d3d73d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.389250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6f313-e6b5-4bb4-a57a-5e00faae06b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.390010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f313-e6b5-4bb4-a57a-5e00faae06b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.390317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f313-e6b5-4bb4-a57a-5e00faae06b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.390868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f313-e6b5-4bb4-a57a-5e00faae06b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.391367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6f313-e6b5-4bb4-a57a-5e00faae06b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:46.393553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6f313-e6b5-4bb4-a57a-5e00faae06b7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.404744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875591ff-0b2f-4307-9081-a152a0d3d73d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.412056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4595b4-2564-4f11-a14c-4f787fc67405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.412868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4595b4-2564-4f11-a14c-4f787fc67405 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.413155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4595b4-2564-4f11-a14c-4f787fc67405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.413591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4595b4-2564-4f11-a14c-4f787fc67405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.415138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4595b4-2564-4f11-a14c-4f787fc67405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:46.417114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4595b4-2564-4f11-a14c-4f787fc67405 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.433830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 899/1788] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:46.436678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4595b4-2564-4f11-a14c-4f787fc67405 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.449502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4595b4-2564-4f11-a14c-4f787fc67405 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.462676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d9d1-27b2-4131-bff6-92f252521ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.468690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d9d1-27b2-4131-bff6-92f252521ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.468690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d9d1-27b2-4131-bff6-92f252521ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.468690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d9d1-27b2-4131-bff6-92f252521ab7 admin admin] PATH_INFO: `/v3/projects/c29d54e57cc74b8b826640ba8da3c95d/users/65b2e405c9334dbda6e889f3025b5912/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.468690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338d9d1-27b2-4131-bff6-92f252521ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29d54e57cc74b8b826640ba8da3c95d, user_id=65b2e405c9334dbda6e889f3025b5912, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:46.471275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338d9d1-27b2-4131-bff6-92f252521ab7 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.493056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 900/1789] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:46 2022] PUT /identity/v3/projects/c29d54e57cc74b8b826640ba8da3c95d/users/65b2e405c9334dbda6e889f3025b5912/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:46.494355 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb9c93-2b6f-4da3-8594-741b692d7bc3 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.497453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb9c93-2b6f-4da3-8594-741b692d7bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.497453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb9c93-2b6f-4da3-8594-741b692d7bc3 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.497453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 901/1790] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:25:46.509304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61143b5-cbbc-46bf-a0bf-d71631001b9d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.509738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61143b5-cbbc-46bf-a0bf-d71631001b9d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.510141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61143b5-cbbc-46bf-a0bf-d71631001b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.530885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 890/1791] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:46 2022] POST /identity/v3/users => generated 284 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:46.532164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0674-e8db-4743-978f-1594bcda9211 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.532271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e61143b5-cbbc-46bf-a0bf-d71631001b9d None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:46.537770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0674-e8db-4743-978f-1594bcda9211 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.537770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0674-e8db-4743-978f-1594bcda9211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.537991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e61143b5-cbbc-46bf-a0bf-d71631001b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:46.590634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-53af0674-e8db-4743-978f-1594bcda9211 None None] MFA Rules not processed for user `65b2e405c9334dbda6e889f3025b5912`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:46.591950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-53af0674-e8db-4743-978f-1594bcda9211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:46.653981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 902/1792] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:25:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:46.655834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61143b5-cbbc-46bf-a0bf-d71631001b9d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.695646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 891/1793] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:46.745542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af0674-e8db-4743-978f-1594bcda9211 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.748268 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-53af0674-e8db-4743-978f-1594bcda9211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:46.811649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61143b5-cbbc-46bf-a0bf-d71631001b9d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.836149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f93c-07aa-466f-8011-81bd33117cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.843276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f93c-07aa-466f-8011-81bd33117cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.845995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f93c-07aa-466f-8011-81bd33117cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.846309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f93c-07aa-466f-8011-81bd33117cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.847085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3f93c-07aa-466f-8011-81bd33117cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:46.855166 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3f93c-07aa-466f-8011-81bd33117cdc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.886233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 903/1794] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:46.890821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f93c-07aa-466f-8011-81bd33117cdc admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.922199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f93c-07aa-466f-8011-81bd33117cdc admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.926642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af0674-e8db-4743-978f-1594bcda9211 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.935501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44973b5-3ca6-485c-a4df-a96fa5a14148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.936385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44973b5-3ca6-485c-a4df-a96fa5a14148 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.936603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44973b5-3ca6-485c-a4df-a96fa5a14148 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.936891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44973b5-3ca6-485c-a4df-a96fa5a14148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.946659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd9b30-671c-43e6-88e9-4ebfbe0c5b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.947350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd9b30-671c-43e6-88e9-4ebfbe0c5b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.947633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd9b30-671c-43e6-88e9-4ebfbe0c5b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.947909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd9b30-671c-43e6-88e9-4ebfbe0c5b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:46.948632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd9b30-671c-43e6-88e9-4ebfbe0c5b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:46.955504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd9b30-671c-43e6-88e9-4ebfbe0c5b75 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.959436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44973b5-3ca6-485c-a4df-a96fa5a14148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:46.962317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44973b5-3ca6-485c-a4df-a96fa5a14148 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:46.974363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 904/1795] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:46.978524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd9b30-671c-43e6-88e9-4ebfbe0c5b75 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:46.988808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 892/1796] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:46.990975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd9b30-671c-43e6-88e9-4ebfbe0c5b75 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:46.996821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:46.998092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:46.998394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:46.998669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] PATH_INFO: `/v3/projects/291c7b7b72a64c999831ca6840f1738e/users/995f8128c8d540b78f6d38168da0a4a3/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:47.019439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=291c7b7b72a64c999831ca6840f1738e, user_id=995f8128c8d540b78f6d38168da0a4a3, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:47.022428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:47.156479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 905/1797] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:46 2022] PUT /identity/v3/projects/291c7b7b72a64c999831ca6840f1738e/users/995f8128c8d540b78f6d38168da0a4a3/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:47.168669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665c428-f76e-4925-9d1c-5462427f6903 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:47.168969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665c428-f76e-4925-9d1c-5462427f6903 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:47.169253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665c428-f76e-4925-9d1c-5462427f6903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:47.208256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:47.210848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:47.236334 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5665c428-f76e-4925-9d1c-5462427f6903 None None] MFA Rules not processed for user `995f8128c8d540b78f6d38168da0a4a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:47.237553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5665c428-f76e-4925-9d1c-5462427f6903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:47.373207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 893/1798] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:47.380586 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665c428-f76e-4925-9d1c-5462427f6903 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:47.421889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6d930-d156-4494-bfd3-1f0ab7151aa2 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:47.434213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98b2d3-bae0-4c77-a525-141ae16b1b8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:47.435237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b2d3-bae0-4c77-a525-141ae16b1b8a service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:47.435531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b2d3-bae0-4c77-a525-141ae16b1b8a service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:47.435817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b2d3-bae0-4c77-a525-141ae16b1b8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:47.451763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98b2d3-bae0-4c77-a525-141ae16b1b8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:47.465684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98b2d3-bae0-4c77-a525-141ae16b1b8a service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:47.491346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 906/1799] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:47.550913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665c428-f76e-4925-9d1c-5462427f6903 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:47.563203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fbbdc-9990-4591-b507-2a6068ed0639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:47.563885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fbbdc-9990-4591-b507-2a6068ed0639 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:47.564170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fbbdc-9990-4591-b507-2a6068ed0639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:47.564449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fbbdc-9990-4591-b507-2a6068ed0639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:47.565150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fbbdc-9990-4591-b507-2a6068ed0639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:47.567382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fbbdc-9990-4591-b507-2a6068ed0639 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:47.782682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 894/1800] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:47 2022] POST /identity/v3/projects => generated 369 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:47.786929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98b2d3-bae0-4c77-a525-141ae16b1b8a service nova] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:47.799621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98b2d3-bae0-4c77-a525-141ae16b1b8a service nova] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:47.864786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069f4f-2841-4d92-a615-1e5f59459b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:47.865568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069f4f-2841-4d92-a615-1e5f59459b2d admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:47.866000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069f4f-2841-4d92-a615-1e5f59459b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:47.866376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069f4f-2841-4d92-a615-1e5f59459b2d admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:47.867179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069f4f-2841-4d92-a615-1e5f59459b2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:47.869322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069f4f-2841-4d92-a615-1e5f59459b2d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:48.046798 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 907/1801] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:47 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:48.074664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fbbdc-9990-4591-b507-2a6068ed0639 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:48.090897 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fbbdc-9990-4591-b507-2a6068ed0639 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:48.096951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe74258-8832-4e2a-84b1-455ea74e395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:48.098413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe74258-8832-4e2a-84b1-455ea74e395f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:48.098852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe74258-8832-4e2a-84b1-455ea74e395f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:48.099280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe74258-8832-4e2a-84b1-455ea74e395f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:48.100232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe74258-8832-4e2a-84b1-455ea74e395f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:48.102578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe74258-8832-4e2a-84b1-455ea74e395f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:48.117860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 895/1802] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:48.122276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069f4f-2841-4d92-a615-1e5f59459b2d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:48.139344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069f4f-2841-4d92-a615-1e5f59459b2d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:48.145964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1354b1-81e5-4163-9242-2124ebd821fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:48.147709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1354b1-81e5-4163-9242-2124ebd821fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:48.148218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1354b1-81e5-4163-9242-2124ebd821fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:48.149462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1354b1-81e5-4163-9242-2124ebd821fb admin admin] PATH_INFO: `/v3/projects/ce617bdcd74647f0956aa7da48f7ec77/users/7491a6ac9d5f4d0fb2e2506818d53c9a/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:48.437069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1354b1-81e5-4163-9242-2124ebd821fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce617bdcd74647f0956aa7da48f7ec77, user_id=7491a6ac9d5f4d0fb2e2506818d53c9a, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:48.454345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1354b1-81e5-4163-9242-2124ebd821fb admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:48.554401 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 908/1803] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:48 2022] PUT /identity/v3/projects/ce617bdcd74647f0956aa7da48f7ec77/users/7491a6ac9d5f4d0fb2e2506818d53c9a/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:48.559399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe74258-8832-4e2a-84b1-455ea74e395f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:48.575518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe74258-8832-4e2a-84b1-455ea74e395f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:48.712637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4a704-66b2-49bc-b62d-2003c400e7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:48.713287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4a704-66b2-49bc-b62d-2003c400e7cc admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:48.713573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4a704-66b2-49bc-b62d-2003c400e7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:48.713878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4a704-66b2-49bc-b62d-2003c400e7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:48.714811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4a704-66b2-49bc-b62d-2003c400e7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:48.717134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4a704-66b2-49bc-b62d-2003c400e7cc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:48.729876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 896/1804] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:48.734717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1354b1-81e5-4163-9242-2124ebd821fb admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:48.763345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1354b1-81e5-4163-9242-2124ebd821fb admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:48.793008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c865a-c569-44a2-88dd-d8265e92d9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:48.797325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c865a-c569-44a2-88dd-d8265e92d9f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:48.798715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c865a-c569-44a2-88dd-d8265e92d9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:48.799314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c865a-c569-44a2-88dd-d8265e92d9f2 admin admin] PATH_INFO: `/v3/projects/ce617bdcd74647f0956aa7da48f7ec77/users/7491a6ac9d5f4d0fb2e2506818d53c9a/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:48.806779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c865a-c569-44a2-88dd-d8265e92d9f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce617bdcd74647f0956aa7da48f7ec77, user_id=7491a6ac9d5f4d0fb2e2506818d53c9a, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:48.811226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c865a-c569-44a2-88dd-d8265e92d9f2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:48.833883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 909/1805] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:48 2022] PUT /identity/v3/projects/ce617bdcd74647f0956aa7da48f7ec77/users/7491a6ac9d5f4d0fb2e2506818d53c9a/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:48.843652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38633fe-bb42-4a76-8804-9a117e16e066 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:48.844538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38633fe-bb42-4a76-8804-9a117e16e066 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:48.844976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38633fe-bb42-4a76-8804-9a117e16e066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:48.874604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d38633fe-bb42-4a76-8804-9a117e16e066 None None] MFA Rules not processed for user `7491a6ac9d5f4d0fb2e2506818d53c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:48.875849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d38633fe-bb42-4a76-8804-9a117e16e066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:48.908373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c865a-c569-44a2-88dd-d8265e92d9f2 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:48.970807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 897/1806] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:48.970807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38633fe-bb42-4a76-8804-9a117e16e066 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:48.970807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d38633fe-bb42-4a76-8804-9a117e16e066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:48.996661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c865a-c569-44a2-88dd-d8265e92d9f2 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:49.001911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:49.003191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:49.003490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:49.003770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] PATH_INFO: `/v3/users/5cd0887121e7424ebd9440df1777f7e3` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:49.004991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd0887121e7424ebd9440df1777f7e3)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:49.006946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:49.044040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38633fe-bb42-4a76-8804-9a117e16e066 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:49.050234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:49.051124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:49.051410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:49.051731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] PATH_INFO: `/v3/users/bb70755b1a4d441fbe11fc2565bc7ae8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:49.052950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb70755b1a4d441fbe11fc2565bc7ae8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:49.055190 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:49.188178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd0887121e7424ebd9440df1777f7e3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.188858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd0887121e7424ebd9440df1777f7e3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.250444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb70755b1a4d441fbe11fc2565bc7ae8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.251085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb70755b1a4d441fbe11fc2565bc7ae8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.251727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb70755b1a4d441fbe11fc2565bc7ae8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.258541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb70755b1a4d441fbe11fc2565bc7ae8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.278396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd0887121e7424ebd9440df1777f7e3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.285426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd0887121e7424ebd9440df1777f7e3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.286127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd0887121e7424ebd9440df1777f7e3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.302366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 910/1807] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:48 2022] DELETE /identity/v3/users/5cd0887121e7424ebd9440df1777f7e3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:49.303162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:49.304474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:49.359178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4af08-afcd-42a3-8eef-53a8c63e9898 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:49.362865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddb411-2378-4a33-9eb4-124c30ad7917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:49.363521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddb411-2378-4a33-9eb4-124c30ad7917 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:49.363805 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddb411-2378-4a33-9eb4-124c30ad7917 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:49.364077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddb411-2378-4a33-9eb4-124c30ad7917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:49.365010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b7a729fe-ba69-4d99-a244-2ad4e62a7f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb70755b1a4d441fbe11fc2565bc7ae8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:49.366011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-67ddb411-2378-4a33-9eb4-124c30ad7917 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:49.406303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 898/1808] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:48 2022] DELETE /identity/v3/users/bb70755b1a4d441fbe11fc2565bc7ae8 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:49.408057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f729e81-ccb1-40a7-af08-0d942d2e199c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:49.408544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f729e81-ccb1-40a7-af08-0d942d2e199c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:49.408897 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f729e81-ccb1-40a7-af08-0d942d2e199c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:49.435150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6f729e81-ccb1-40a7-af08-0d942d2e199c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:49.436634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6f729e81-ccb1-40a7-af08-0d942d2e199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:49.447687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddb411-2378-4a33-9eb4-124c30ad7917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:49.449867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddb411-2378-4a33-9eb4-124c30ad7917 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:49.461965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 911/1809] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:49 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:49.463359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d224a-bca9-4460-bdfd-b2ca84960e51 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:49.463650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d224a-bca9-4460-bdfd-b2ca84960e51 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:49.463923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d224a-bca9-4460-bdfd-b2ca84960e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:49.486817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-060d224a-bca9-4460-bdfd-b2ca84960e51 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:49.488217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-060d224a-bca9-4460-bdfd-b2ca84960e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:49.535792 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 899/1810] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:49.548413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f729e81-ccb1-40a7-af08-0d942d2e199c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:49.550059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6f729e81-ccb1-40a7-af08-0d942d2e199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:49.588828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 912/1811] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:49.598786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d224a-bca9-4460-bdfd-b2ca84960e51 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:49.600227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-060d224a-bca9-4460-bdfd-b2ca84960e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:49.646740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f729e81-ccb1-40a7-af08-0d942d2e199c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:49.656094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:49.656942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:49.658746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:49.659134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:49.686654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:49.691255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:49.711172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d224a-bca9-4460-bdfd-b2ca84960e51 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:49.716640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:49.717479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:49.717766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:49.718070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:49.719498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 900/1812] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:49.729662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:49.731952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:49.743811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 913/1813] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:50.769044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:50.772606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:50.803482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b1658-e504-409b-9ecf-e6df67274993 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:50.808687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:50.809616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:50.810000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:50.810357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] PATH_INFO: `/v3/projects/1331e21bd7ce4ffba3566d44c8342f7f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:50.811606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=1331e21bd7ce4ffba3566d44c8342f7f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:50.814595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:50.900397 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:50.901875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:50.916566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580a450-dc0c-40dd-8c8d-a58db0e0be5f service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:50.939716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1331e21bd7ce4ffba3566d44c8342f7f', 'request_id': 'req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:50.941439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:50.943560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:50.943780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:50.944057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] PATH_INFO: `/v3/projects/5590fd98524b44eb8d71dc59fefa6993` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:50.946252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5590fd98524b44eb8d71dc59fefa6993)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:50.946576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1331e21bd7ce4ffba3566d44c8342f7f', 'request_id': 'req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:50.948599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:51.009587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1331e21bd7ce4ffba3566d44c8342f7f', 'request_id': 'req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.021479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] The token cache is being invalidate because project 1331e21bd7ce4ffba3566d44c8342f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:51.021779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.022464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.024054 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 901/1814] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:50 2022] DELETE /identity/v3/projects/1331e21bd7ce4ffba3566d44c8342f7f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:51.028353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:51.029695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:51.097307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5590fd98524b44eb8d71dc59fefa6993', 'request_id': 'req-538af80b-558c-4a11-a716-a9a1342a5607'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.106893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5590fd98524b44eb8d71dc59fefa6993', 'request_id': 'req-538af80b-558c-4a11-a716-a9a1342a5607'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.141724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d0e8d-2981-40f0-8dd8-584c0f6fb5ae admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:51.153482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5590fd98524b44eb8d71dc59fefa6993', 'request_id': 'req-538af80b-558c-4a11-a716-a9a1342a5607'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.168567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] The token cache is being invalidate because project 5590fd98524b44eb8d71dc59fefa6993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:51.168861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.169541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.174406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 914/1815] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:50 2022] DELETE /identity/v3/projects/5590fd98524b44eb8d71dc59fefa6993 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:51.178337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:51.179769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:51.210887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:51.218579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:51.218579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:51.218579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] PATH_INFO: `/v3/users/65b2e405c9334dbda6e889f3025b5912` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:51.218579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b2e405c9334dbda6e889f3025b5912)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:51.219539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:51.262449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af80b-558c-4a11-a716-a9a1342a5607 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:51.268381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:51.269265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:51.269545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:51.269841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] PATH_INFO: `/v3/users/b848f524d8a14915965284767139a4a2` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:51.271336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b848f524d8a14915965284767139a4a2)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:51.273287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:51.490029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b2e405c9334dbda6e889f3025b5912'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.490869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b2e405c9334dbda6e889f3025b5912'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.492149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b2e405c9334dbda6e889f3025b5912'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.504614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b2e405c9334dbda6e889f3025b5912'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.587228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b2e405c9334dbda6e889f3025b5912'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.600167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b848f524d8a14915965284767139a4a2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.601737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b848f524d8a14915965284767139a4a2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.625908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 902/1816] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:51 2022] DELETE /identity/v3/users/65b2e405c9334dbda6e889f3025b5912 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:51.683033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b848f524d8a14915965284767139a4a2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.696859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b848f524d8a14915965284767139a4a2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.697597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b848f524d8a14915965284767139a4a2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:51.718147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 915/1817] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:51 2022] DELETE /identity/v3/users/b848f524d8a14915965284767139a4a2 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:52.235158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:52.236511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:52.239062 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:52.239969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:52.344577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554eb84-3b77-42e0-b812-8bd9e5947b2a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:52.349742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f363a4-7473-4d62-825a-53997634f5e3 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:52.351940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:52.351940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.351940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.351940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] PATH_INFO: `/v3/projects/c29d54e57cc74b8b826640ba8da3c95d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.352831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29d54e57cc74b8b826640ba8da3c95d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:52.354962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:52.355145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:52.356539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.356954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.357136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] PATH_INFO: `/v3/projects/89cb9b363e4a4d6b913dea40bdbe07b6` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.358386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=89cb9b363e4a4d6b913dea40bdbe07b6)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:52.360994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:52.524023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29d54e57cc74b8b826640ba8da3c95d', 'request_id': 'req-bbdbbf30-0563-4736-aebf-53980ad50415'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.531316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29d54e57cc74b8b826640ba8da3c95d', 'request_id': 'req-bbdbbf30-0563-4736-aebf-53980ad50415'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.552093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89cb9b363e4a4d6b913dea40bdbe07b6', 'request_id': 'req-950dbd58-29e4-474d-9bb9-744f9e034d3a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.561202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89cb9b363e4a4d6b913dea40bdbe07b6', 'request_id': 'req-950dbd58-29e4-474d-9bb9-744f9e034d3a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.633086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29d54e57cc74b8b826640ba8da3c95d', 'request_id': 'req-bbdbbf30-0563-4736-aebf-53980ad50415'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.645274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] The token cache is being invalidate because project c29d54e57cc74b8b826640ba8da3c95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:52.645673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.646458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bbdbbf30-0563-4736-aebf-53980ad50415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.648155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 903/1818] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:52 2022] DELETE /identity/v3/projects/c29d54e57cc74b8b826640ba8da3c95d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:52.656773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f729-fb3c-464b-969b-f1c4122b320a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.657293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f729-fb3c-464b-969b-f1c4122b320a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.658047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f729-fb3c-464b-969b-f1c4122b320a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.687958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3b32f729-fb3c-464b-969b-f1c4122b320a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:52.690337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3b32f729-fb3c-464b-969b-f1c4122b320a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:52.757446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 904/1819] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:52.762669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89cb9b363e4a4d6b913dea40bdbe07b6', 'request_id': 'req-950dbd58-29e4-474d-9bb9-744f9e034d3a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.766758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b333-45fd-4b2e-986c-d6c13da48632 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.768344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b333-45fd-4b2e-986c-d6c13da48632 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.768638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b333-45fd-4b2e-986c-d6c13da48632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.782779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] The token cache is being invalidate because project 89cb9b363e4a4d6b913dea40bdbe07b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:52.784297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.785310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-950dbd58-29e4-474d-9bb9-744f9e034d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:52.787791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 916/1820] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:52 2022] DELETE /identity/v3/projects/89cb9b363e4a4d6b913dea40bdbe07b6 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:52.790948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b421b333-45fd-4b2e-986c-d6c13da48632 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:52.792363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b421b333-45fd-4b2e-986c-d6c13da48632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:52.792482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226845c-8744-488e-8a69-02a457b02c8d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.792800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226845c-8744-488e-8a69-02a457b02c8d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.793085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226845c-8744-488e-8a69-02a457b02c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.804299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 905/1821] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:52.812646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b333-45fd-4b2e-986c-d6c13da48632 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:52.826809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b333-45fd-4b2e-986c-d6c13da48632 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:52.832468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:52.833357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.833775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.834241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.835115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:52.836294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:52.838630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:52.853641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 906/1822] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:52.857205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1226845c-8744-488e-8a69-02a457b02c8d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:52.857205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1226845c-8744-488e-8a69-02a457b02c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:52.859384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:52.869628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59357058-fc06-4ded-88ca-063eba7c504d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:52.870437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 917/1823] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:52.878804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40c5ad-0b72-4d87-85c4-0f8ef4045535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:52.878982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44caeb4-f7c1-4414-9eea-1cdb8c5f1e70 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.879263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44caeb4-f7c1-4414-9eea-1cdb8c5f1e70 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.879479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c5ad-0b72-4d87-85c4-0f8ef4045535 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.879678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44caeb4-f7c1-4414-9eea-1cdb8c5f1e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.879766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c5ad-0b72-4d87-85c4-0f8ef4045535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.880031 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c5ad-0b72-4d87-85c4-0f8ef4045535 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.880726 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40c5ad-0b72-4d87-85c4-0f8ef4045535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:52.882704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40c5ad-0b72-4d87-85c4-0f8ef4045535 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:52.899856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b44caeb4-f7c1-4414-9eea-1cdb8c5f1e70 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:52.900590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b44caeb4-f7c1-4414-9eea-1cdb8c5f1e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:52.909625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 918/1824] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:52.918434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44caeb4-f7c1-4414-9eea-1cdb8c5f1e70 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:52.934234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44caeb4-f7c1-4414-9eea-1cdb8c5f1e70 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:52.940833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:52.946720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.946720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.946720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.946720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:52.946720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:52.946720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:52.957661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 919/1825] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:25:52.961743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:52.971923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87864a5e-93f4-4ee2-ac09-99799cd8f2c2 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:52.976789 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b90cf-1cce-4826-bd94-fb8213ce3989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:52.977434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b90cf-1cce-4826-bd94-fb8213ce3989 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:52.977722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b90cf-1cce-4826-bd94-fb8213ce3989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:52.978030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b90cf-1cce-4826-bd94-fb8213ce3989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:52.978791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b90cf-1cce-4826-bd94-fb8213ce3989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:52.980750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b90cf-1cce-4826-bd94-fb8213ce3989 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.075505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 907/1826] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:52 2022] POST /identity/v3/projects => generated 381 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:53.079517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40c5ad-0b72-4d87-85c4-0f8ef4045535 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.140659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 920/1827] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:52 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:53.141813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b90cf-1cce-4826-bd94-fb8213ce3989 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.144081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40c5ad-0b72-4d87-85c4-0f8ef4045535 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.207239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b90cf-1cce-4826-bd94-fb8213ce3989 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.211941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12b652-6516-4c07-bb3d-9063f181ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.212617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12b652-6516-4c07-bb3d-9063f181ac85 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.212903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12b652-6516-4c07-bb3d-9063f181ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.213224 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12b652-6516-4c07-bb3d-9063f181ac85 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.213987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12b652-6516-4c07-bb3d-9063f181ac85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.216008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12b652-6516-4c07-bb3d-9063f181ac85 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.220575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.221252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.221569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.221880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.222570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.224512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.318935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 908/1828] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:53 2022] POST /identity/v3/users => generated 291 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:53.319390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 921/1829] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:53 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:53.319752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.319752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:53.322527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12b652-6516-4c07-bb3d-9063f181ac85 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.331739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d45b7-0533-456a-82f6-2118dc315b85 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.334909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12b652-6516-4c07-bb3d-9063f181ac85 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.336713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.340609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05513880-dad9-4dc3-967c-95a618584f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.341309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05513880-dad9-4dc3-967c-95a618584f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.341927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05513880-dad9-4dc3-967c-95a618584f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.342395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05513880-dad9-4dc3-967c-95a618584f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.343138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05513880-dad9-4dc3-967c-95a618584f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.344754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05513880-dad9-4dc3-967c-95a618584f92 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.345854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.346154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.347977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] PATH_INFO: `/v3/users/995f8128c8d540b78f6d38168da0a4a3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.347977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=995f8128c8d540b78f6d38168da0a4a3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.349758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.372455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 922/1830] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:53.375734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05513880-dad9-4dc3-967c-95a618584f92 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.400134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05513880-dad9-4dc3-967c-95a618584f92 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.411844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6695896-deec-4f70-9e13-456c6f947b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.412526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6695896-deec-4f70-9e13-456c6f947b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.412814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6695896-deec-4f70-9e13-456c6f947b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.413084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6695896-deec-4f70-9e13-456c6f947b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.413829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6695896-deec-4f70-9e13-456c6f947b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.415450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6695896-deec-4f70-9e13-456c6f947b47 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.437420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 923/1831] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:53.438766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6695896-deec-4f70-9e13-456c6f947b47 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.449413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6695896-deec-4f70-9e13-456c6f947b47 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.470925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f12831-3190-4bb4-9196-0a3009e2b2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.471616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f12831-3190-4bb4-9196-0a3009e2b2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.479371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f12831-3190-4bb4-9196-0a3009e2b2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.479371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f12831-3190-4bb4-9196-0a3009e2b2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.479371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f12831-3190-4bb4-9196-0a3009e2b2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.479371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f12831-3190-4bb4-9196-0a3009e2b2c8 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.490017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 924/1832] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:53.491015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f12831-3190-4bb4-9196-0a3009e2b2c8 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.506160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f12831-3190-4bb4-9196-0a3009e2b2c8 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.512185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4d79b-9289-4dbb-8465-60be13422385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.512868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4d79b-9289-4dbb-8465-60be13422385 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.513150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4d79b-9289-4dbb-8465-60be13422385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.513564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4d79b-9289-4dbb-8465-60be13422385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.514383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4d79b-9289-4dbb-8465-60be13422385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.516026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f4d79b-9289-4dbb-8465-60be13422385 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.535995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 925/1833] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:53.535995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4d79b-9289-4dbb-8465-60be13422385 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.546766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4d79b-9289-4dbb-8465-60be13422385 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.553100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.554129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.554454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.554740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] PATH_INFO: `/v3/projects/ba1e3fb2ab0443e6bbe2f304a60cd709/users/bac2de2db4394a5193f9ad9a3c02816b/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.574598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1e3fb2ab0443e6bbe2f304a60cd709, user_id=bac2de2db4394a5193f9ad9a3c02816b, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.584247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.606665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '995f8128c8d540b78f6d38168da0a4a3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:53.607384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '995f8128c8d540b78f6d38168da0a4a3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:53.608006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '995f8128c8d540b78f6d38168da0a4a3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:53.615513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '995f8128c8d540b78f6d38168da0a4a3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:53.672037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 926/1834] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:53 2022] PUT /identity/v3/projects/ba1e3fb2ab0443e6bbe2f304a60cd709/users/bac2de2db4394a5193f9ad9a3c02816b/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:53.672863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.674528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:53.723401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-22175612-8249-41fd-b419-c6ee661604b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '995f8128c8d540b78f6d38168da0a4a3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:53.743306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 909/1835] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:53 2022] DELETE /identity/v3/users/995f8128c8d540b78f6d38168da0a4a3 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:53.745012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6e44-a437-4763-b55b-4b77082f8835 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.745389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6e44-a437-4763-b55b-4b77082f8835 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.745594 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d6e44-a437-4763-b55b-4b77082f8835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.748823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c84be1-d0f4-438c-b67d-2cda9d084792 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.778489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-887d6e44-a437-4763-b55b-4b77082f8835 None None] MFA Rules not processed for user `bac2de2db4394a5193f9ad9a3c02816b`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:53.780431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-887d6e44-a437-4763-b55b-4b77082f8835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:53.842408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.843610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.843744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.844178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] PATH_INFO: `/v3/projects/4158957ed8a8403b943babd5fd3d3dee/users/56a5ea192dcd496c803e84ded4c99ba0/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.859844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 910/1836] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:53.861162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f0c6f-abc3-434b-b7f1-52f25fc7da6a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.861453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f0c6f-abc3-434b-b7f1-52f25fc7da6a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.861732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f0c6f-abc3-434b-b7f1-52f25fc7da6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.862778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4158957ed8a8403b943babd5fd3d3dee, user_id=56a5ea192dcd496c803e84ded4c99ba0, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.865690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.878982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-208f0c6f-abc3-434b-b7f1-52f25fc7da6a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:53.880186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-208f0c6f-abc3-434b-b7f1-52f25fc7da6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:53.887853 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 911/1837] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:53.892847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f0c6f-abc3-434b-b7f1-52f25fc7da6a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.892847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-208f0c6f-abc3-434b-b7f1-52f25fc7da6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:53.901240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f0c6f-abc3-434b-b7f1-52f25fc7da6a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:53.906531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf7db0-926e-4f43-ac98-ca187651714b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:53.907122 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf7db0-926e-4f43-ac98-ca187651714b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:53.907408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf7db0-926e-4f43-ac98-ca187651714b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:53.907686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf7db0-926e-4f43-ac98-ca187651714b admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:53.908372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf7db0-926e-4f43-ac98-ca187651714b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:53.910346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf7db0-926e-4f43-ac98-ca187651714b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:53.969841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 927/1838] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:53 2022] PUT /identity/v3/projects/4158957ed8a8403b943babd5fd3d3dee/users/56a5ea192dcd496c803e84ded4c99ba0/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:53.969841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:53.969841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:53.999277 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 912/1839] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:53 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:54.000950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff09d2-2355-4e29-93ed-cb61a910d0b4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.001472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff09d2-2355-4e29-93ed-cb61a910d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.002301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff09d2-2355-4e29-93ed-cb61a910d0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.038652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1dff09d2-2355-4e29-93ed-cb61a910d0b4 None None] MFA Rules not processed for user `56a5ea192dcd496c803e84ded4c99ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:54.039932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1f72b-662d-44b9-9053-8b24a0942895 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.040089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1dff09d2-2355-4e29-93ed-cb61a910d0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:54.100404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58be05-40d9-428f-bddc-d78756ecbcd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.101125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58be05-40d9-428f-bddc-d78756ecbcd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.101365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58be05-40d9-428f-bddc-d78756ecbcd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.101813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58be05-40d9-428f-bddc-d78756ecbcd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.114498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 913/1840] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:54.115449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff09d2-2355-4e29-93ed-cb61a910d0b4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.178905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff09d2-2355-4e29-93ed-cb61a910d0b4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.180328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58be05-40d9-428f-bddc-d78756ecbcd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.181823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58be05-40d9-428f-bddc-d78756ecbcd0 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.184132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f36b6b-1b5d-4ba6-9aff-9f893bc09faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.184806 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f36b6b-1b5d-4ba6-9aff-9f893bc09faa admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.185094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f36b6b-1b5d-4ba6-9aff-9f893bc09faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.185376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f36b6b-1b5d-4ba6-9aff-9f893bc09faa admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.186100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f36b6b-1b5d-4ba6-9aff-9f893bc09faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.188654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f36b6b-1b5d-4ba6-9aff-9f893bc09faa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.193007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 928/1841] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:54.193824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58be05-40d9-428f-bddc-d78756ecbcd0 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.203018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58be05-40d9-428f-bddc-d78756ecbcd0 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.207774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdef65-c019-4341-8874-4cc194af072c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.208429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdef65-c019-4341-8874-4cc194af072c admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.208720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdef65-c019-4341-8874-4cc194af072c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.213591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdef65-c019-4341-8874-4cc194af072c admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.213591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdef65-c019-4341-8874-4cc194af072c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.213591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdef65-c019-4341-8874-4cc194af072c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.323360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 914/1842] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:54 2022] POST /identity/v3/users => generated 291 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:54.327286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f36b6b-1b5d-4ba6-9aff-9f893bc09faa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.336881 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f36b6b-1b5d-4ba6-9aff-9f893bc09faa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.341693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd88eb9-8651-41a9-8d90-9a7d3a4ecc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.342362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd88eb9-8651-41a9-8d90-9a7d3a4ecc7f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.342675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd88eb9-8651-41a9-8d90-9a7d3a4ecc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.342949 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd88eb9-8651-41a9-8d90-9a7d3a4ecc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.343671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd88eb9-8651-41a9-8d90-9a7d3a4ecc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.345293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd88eb9-8651-41a9-8d90-9a7d3a4ecc7f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.354615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 929/1843] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:54 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:54.364747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 915/1844] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:54.365865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd88eb9-8651-41a9-8d90-9a7d3a4ecc7f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.366185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdef65-c019-4341-8874-4cc194af072c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.376855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd88eb9-8651-41a9-8d90-9a7d3a4ecc7f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.427631 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdef65-c019-4341-8874-4cc194af072c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.431900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2ee95-75ee-44a0-8cfb-dd8d2b985c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.432293 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcc196-6f64-41d5-a663-36dc511d204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.432518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ee95-75ee-44a0-8cfb-dd8d2b985c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.432809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ee95-75ee-44a0-8cfb-dd8d2b985c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.433092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ee95-75ee-44a0-8cfb-dd8d2b985c89 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.433217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcc196-6f64-41d5-a663-36dc511d204c admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.434059 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcc196-6f64-41d5-a663-36dc511d204c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.434341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcc196-6f64-41d5-a663-36dc511d204c admin admin] PATH_INFO: `/v3/projects/1840529b8cda4a8ea0790e1f854c6263/users/5bc73b03f26a426cbdf7a6b8ccc08d86/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.434444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2ee95-75ee-44a0-8cfb-dd8d2b985c89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.436329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2ee95-75ee-44a0-8cfb-dd8d2b985c89 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.449329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcc196-6f64-41d5-a663-36dc511d204c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1840529b8cda4a8ea0790e1f854c6263, user_id=5bc73b03f26a426cbdf7a6b8ccc08d86, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.452014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcc196-6f64-41d5-a663-36dc511d204c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.576863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 930/1845] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:54 2022] PUT /identity/v3/projects/1840529b8cda4a8ea0790e1f854c6263/users/5bc73b03f26a426cbdf7a6b8ccc08d86/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:54.581859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 916/1846] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:54 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:54.582000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcc196-6f64-41d5-a663-36dc511d204c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.583456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2ee95-75ee-44a0-8cfb-dd8d2b985c89 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.594374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2ee95-75ee-44a0-8cfb-dd8d2b985c89 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.643241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcc196-6f64-41d5-a663-36dc511d204c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.648098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b07bf9-7ad2-40ac-9942-2138f375b3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.648772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b07bf9-7ad2-40ac-9942-2138f375b3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.649057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b07bf9-7ad2-40ac-9942-2138f375b3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.649336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b07bf9-7ad2-40ac-9942-2138f375b3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.650134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b07bf9-7ad2-40ac-9942-2138f375b3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.652552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f4d85-840a-40f1-8b25-0aea1be746d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.652651 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b07bf9-7ad2-40ac-9942-2138f375b3f0 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.653156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f4d85-840a-40f1-8b25-0aea1be746d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.653677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f4d85-840a-40f1-8b25-0aea1be746d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.657631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f4d85-840a-40f1-8b25-0aea1be746d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.657631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f4d85-840a-40f1-8b25-0aea1be746d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.657631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f4d85-840a-40f1-8b25-0aea1be746d6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.662557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 931/1847] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:54.667246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 917/1848] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:54.667690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b07bf9-7ad2-40ac-9942-2138f375b3f0 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.673090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f4d85-840a-40f1-8b25-0aea1be746d6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.682600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b07bf9-7ad2-40ac-9942-2138f375b3f0 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.688408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.689210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f4d85-840a-40f1-8b25-0aea1be746d6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.690163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.690632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.691110 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] PATH_INFO: `/v3/projects/1840529b8cda4a8ea0790e1f854c6263/users/5bc73b03f26a426cbdf7a6b8ccc08d86/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.693820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1840529b8cda4a8ea0790e1f854c6263, user_id=5bc73b03f26a426cbdf7a6b8ccc08d86, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.696094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e49963-ab84-4c76-8612-ce240d9b6998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.697284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.697672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e49963-ab84-4c76-8612-ce240d9b6998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.698008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e49963-ab84-4c76-8612-ce240d9b6998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.698392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e49963-ab84-4c76-8612-ce240d9b6998 admin admin] PATH_INFO: `/v3/projects/806cc8227c184c01afa55ea1d2b61236/users/7b11b6d68be342518e1f1279720bd448/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.717121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 932/1849] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:54 2022] PUT /identity/v3/projects/1840529b8cda4a8ea0790e1f854c6263/users/5bc73b03f26a426cbdf7a6b8ccc08d86/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:54.717121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.720610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:54.726235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e49963-ab84-4c76-8612-ce240d9b6998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=806cc8227c184c01afa55ea1d2b61236, user_id=7b11b6d68be342518e1f1279720bd448, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.729211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e49963-ab84-4c76-8612-ce240d9b6998 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.789366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd021c7-adda-4ac2-b04e-067db74bd256 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:54.794837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:54.795712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.795984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.796262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] PATH_INFO: `/v3/projects/291c7b7b72a64c999831ca6840f1738e` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.797449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=291c7b7b72a64c999831ca6840f1738e)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:54.800451 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:54.811439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 918/1850] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:54 2022] PUT /identity/v3/projects/806cc8227c184c01afa55ea1d2b61236/users/7b11b6d68be342518e1f1279720bd448/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:54.813060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52be8e2-6610-4e6d-bf7f-6e083d5144ac None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:54.813351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52be8e2-6610-4e6d-bf7f-6e083d5144ac None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:54.813795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52be8e2-6610-4e6d-bf7f-6e083d5144ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:54.850119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a52be8e2-6610-4e6d-bf7f-6e083d5144ac None None] MFA Rules not processed for user `5bc73b03f26a426cbdf7a6b8ccc08d86`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:54.850119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a52be8e2-6610-4e6d-bf7f-6e083d5144ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:54.913126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291c7b7b72a64c999831ca6840f1738e', 'request_id': 'req-ae5e574e-4e70-4695-ac59-7da2f5b956d6'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:54.916184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 919/1851] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:54 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:54.917118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52be8e2-6610-4e6d-bf7f-6e083d5144ac None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:54.923929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291c7b7b72a64c999831ca6840f1738e', 'request_id': 'req-ae5e574e-4e70-4695-ac59-7da2f5b956d6'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:54.980725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52be8e2-6610-4e6d-bf7f-6e083d5144ac None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:55.009270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291c7b7b72a64c999831ca6840f1738e', 'request_id': 'req-ae5e574e-4e70-4695-ac59-7da2f5b956d6'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:55.021640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] The token cache is being invalidate because project 291c7b7b72a64c999831ca6840f1738e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:55.021990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:55.022682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:55.024311 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 933/1852] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:54 2022] DELETE /identity/v3/projects/291c7b7b72a64c999831ca6840f1738e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:55.025026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:55.026366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.039135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:55.039810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.040092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.040379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.041145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:55.043276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:55.053881 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 920/1853] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:55.054976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:55.056659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.069712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66448028-f81b-4f44-8237-ccb4a5d6aa22 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:55.075440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:55.076321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.076611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.077155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] PATH_INFO: `/v3/users/7491a6ac9d5f4d0fb2e2506818d53c9a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.078678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7491a6ac9d5f4d0fb2e2506818d53c9a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:55.080105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:55.094197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e574e-4e70-4695-ac59-7da2f5b956d6 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:55.097947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:55.098820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.098904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.099150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.100993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.165899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:55.168142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:55.178009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 934/1854] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:55 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:55.178764 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:55.180240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.193158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95602b6b-1e15-4028-ae80-665ec65a5ff4 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:55.197937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6389c9-dc8b-4d26-aa6b-332e4ff90f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:55.198835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6389c9-dc8b-4d26-aa6b-332e4ff90f75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.199121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6389c9-dc8b-4d26-aa6b-332e4ff90f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.199394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6389c9-dc8b-4d26-aa6b-332e4ff90f75 admin admin] PATH_INFO: `/v3/projects/806cc8227c184c01afa55ea1d2b61236/users/7b11b6d68be342518e1f1279720bd448/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.201338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6389c9-dc8b-4d26-aa6b-332e4ff90f75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=806cc8227c184c01afa55ea1d2b61236, user_id=7b11b6d68be342518e1f1279720bd448, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:55.204063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6389c9-dc8b-4d26-aa6b-332e4ff90f75 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:55.213936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 935/1855] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:55 2022] PUT /identity/v3/projects/806cc8227c184c01afa55ea1d2b61236/users/7b11b6d68be342518e1f1279720bd448/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:55.224435 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2a0a7-1798-4dec-83c7-16f6eeaa3079 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.224727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2a0a7-1798-4dec-83c7-16f6eeaa3079 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.225008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2a0a7-1798-4dec-83c7-16f6eeaa3079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.235938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7491a6ac9d5f4d0fb2e2506818d53c9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:55.236687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7491a6ac9d5f4d0fb2e2506818d53c9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:55.237316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7491a6ac9d5f4d0fb2e2506818d53c9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:55.244282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7491a6ac9d5f4d0fb2e2506818d53c9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:55.252497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f2b2a0a7-1798-4dec-83c7-16f6eeaa3079 None None] MFA Rules not processed for user `7b11b6d68be342518e1f1279720bd448`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:55.253743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f2b2a0a7-1798-4dec-83c7-16f6eeaa3079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.319579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 936/1856] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:55.320678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2a0a7-1798-4dec-83c7-16f6eeaa3079 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:55.322045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f2b2a0a7-1798-4dec-83c7-16f6eeaa3079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.324735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7491a6ac9d5f4d0fb2e2506818d53c9a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:55.343378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 921/1857] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:55 2022] DELETE /identity/v3/users/7491a6ac9d5f4d0fb2e2506818d53c9a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:55.344514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:55.346009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.379347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2a0a7-1798-4dec-83c7-16f6eeaa3079 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:55.411475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee661f99-f687-4a04-bbd3-9ff7a4849a35 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:55.415829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ca09f-ecc0-4341-a39c-b4e5ef9df445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:55.416703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ca09f-ecc0-4341-a39c-b4e5ef9df445 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.416989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ca09f-ecc0-4341-a39c-b4e5ef9df445 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.417266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ca09f-ecc0-4341-a39c-b4e5ef9df445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.550876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:55.551534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.551820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.552091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.554080 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.561314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ca09f-ecc0-4341-a39c-b4e5ef9df445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:55.562789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ca09f-ecc0-4341-a39c-b4e5ef9df445 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:55.572817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 922/1858] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:55.615465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:55.616920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:55.626857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 937/1859] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:55.742331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ca09f-ecc0-4341-a39c-b4e5ef9df445 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:55.751174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ca09f-ecc0-4341-a39c-b4e5ef9df445 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:55.755185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db0a44-664b-4f25-8f16-6da7b4f89432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:55.755846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db0a44-664b-4f25-8f16-6da7b4f89432 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.756130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db0a44-664b-4f25-8f16-6da7b4f89432 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.756431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db0a44-664b-4f25-8f16-6da7b4f89432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.758266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-97db0a44-664b-4f25-8f16-6da7b4f89432 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.816275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db0a44-664b-4f25-8f16-6da7b4f89432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:55.817515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:55.817735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db0a44-664b-4f25-8f16-6da7b4f89432 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:55.818974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:55.829487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 923/1860] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:55.899745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d4a02-3678-4aa1-87fb-0268d0d502d0 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:55.904953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:55.905886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:55.906324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:55.906604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] PATH_INFO: `/v3/projects/ce617bdcd74647f0956aa7da48f7ec77` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:55.907813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce617bdcd74647f0956aa7da48f7ec77)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:55.909751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:56.090440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce617bdcd74647f0956aa7da48f7ec77', 'request_id': 'req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.095633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce617bdcd74647f0956aa7da48f7ec77', 'request_id': 'req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.231766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce617bdcd74647f0956aa7da48f7ec77', 'request_id': 'req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.245656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] The token cache is being invalidate because project ce617bdcd74647f0956aa7da48f7ec77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:56.245999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.246670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0e31e63d-0c0d-4fbb-84a5-335d8b8268f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.248300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 938/1861] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:55 2022] DELETE /identity/v3/projects/ce617bdcd74647f0956aa7da48f7ec77 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:56.253121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50d34b-0382-4b99-a07b-2b9de3470205 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.253413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50d34b-0382-4b99-a07b-2b9de3470205 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.253698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50d34b-0382-4b99-a07b-2b9de3470205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.272965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ea50d34b-0382-4b99-a07b-2b9de3470205 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:56.274211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ea50d34b-0382-4b99-a07b-2b9de3470205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:56.343223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 924/1862] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:56.351894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ebd8c-11b6-44bc-a502-a4298e0f948f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.352190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ebd8c-11b6-44bc-a502-a4298e0f948f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.352470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ebd8c-11b6-44bc-a502-a4298e0f948f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.374321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-741ebd8c-11b6-44bc-a502-a4298e0f948f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:56.375541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-741ebd8c-11b6-44bc-a502-a4298e0f948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:56.376281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50d34b-0382-4b99-a07b-2b9de3470205 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:56.377667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ea50d34b-0382-4b99-a07b-2b9de3470205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:56.383581 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 939/1863] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:56.388064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ebd8c-11b6-44bc-a502-a4298e0f948f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:56.391007 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50d34b-0382-4b99-a07b-2b9de3470205 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:56.396001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:56.396868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.397095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ebd8c-11b6-44bc-a502-a4298e0f948f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:56.397205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.397418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] PATH_INFO: `/v3/users/56a5ea192dcd496c803e84ded4c99ba0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.398694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a5ea192dcd496c803e84ded4c99ba0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:56.400616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:56.401921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:56.403112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.403395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.403675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.404415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:56.404722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:56.406344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:56.417917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 940/1864] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:25:56.427252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:56.437105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4e545-fcff-47be-97d1-97592c93488e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:56.441698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034ed77-8178-4b97-9415-dadc1b5055ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:56.442451 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ed77-8178-4b97-9415-dadc1b5055ed admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.442752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ed77-8178-4b97-9415-dadc1b5055ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.446974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ed77-8178-4b97-9415-dadc1b5055ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.446974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034ed77-8178-4b97-9415-dadc1b5055ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:56.446974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034ed77-8178-4b97-9415-dadc1b5055ed admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:56.623171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 941/1865] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:56 2022] POST /identity/v3/projects => generated 389 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:56.627602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034ed77-8178-4b97-9415-dadc1b5055ed admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:56.689846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a5ea192dcd496c803e84ded4c99ba0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.690578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a5ea192dcd496c803e84ded4c99ba0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.691699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a5ea192dcd496c803e84ded4c99ba0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.693515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034ed77-8178-4b97-9415-dadc1b5055ed admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:56.700060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:56.700697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a5ea192dcd496c803e84ded4c99ba0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.701886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.702825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.703284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.704410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:56.706610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:56.802424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-77da82da-bd14-4e89-a92f-d52ea38ddc23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a5ea192dcd496c803e84ded4c99ba0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:56.822218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 925/1866] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:56 2022] DELETE /identity/v3/users/56a5ea192dcd496c803e84ded4c99ba0 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:56.827246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60441434-a4f7-490f-8d1e-2bc004b3ba14 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.827651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60441434-a4f7-490f-8d1e-2bc004b3ba14 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.828078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60441434-a4f7-490f-8d1e-2bc004b3ba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.831027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 942/1867] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:56 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:56.832032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:56.833563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:56.845219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e44f3-0926-4ab6-acf6-5ecec7dc1375 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:56.846633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-60441434-a4f7-490f-8d1e-2bc004b3ba14 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:56.847841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-60441434-a4f7-490f-8d1e-2bc004b3ba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:56.900330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2da3ba-8d81-4350-8dad-aa7bb3cb401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:56.900927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da3ba-8d81-4350-8dad-aa7bb3cb401e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.901211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da3ba-8d81-4350-8dad-aa7bb3cb401e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.901488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da3ba-8d81-4350-8dad-aa7bb3cb401e admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.903208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2da3ba-8d81-4350-8dad-aa7bb3cb401e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:56.904835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2da3ba-8d81-4350-8dad-aa7bb3cb401e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:56.906088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 926/1868] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:56.917452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 943/1869] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:56.917605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60441434-a4f7-490f-8d1e-2bc004b3ba14 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:56.917605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-60441434-a4f7-490f-8d1e-2bc004b3ba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:56.919003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2da3ba-8d81-4350-8dad-aa7bb3cb401e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:56.928264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2da3ba-8d81-4350-8dad-aa7bb3cb401e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:56.932916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f07b-2a69-46af-8e6d-4edd9c9ebf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:56.933584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f07b-2a69-46af-8e6d-4edd9c9ebf51 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.933899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f07b-2a69-46af-8e6d-4edd9c9ebf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.934180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8f07b-2a69-46af-8e6d-4edd9c9ebf51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.936010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8f07b-2a69-46af-8e6d-4edd9c9ebf51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:56.937716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8f07b-2a69-46af-8e6d-4edd9c9ebf51 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:56.948594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 944/1870] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:56.952556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f07b-2a69-46af-8e6d-4edd9c9ebf51 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:56.961978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8f07b-2a69-46af-8e6d-4edd9c9ebf51 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:56.967000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84be14b-4058-4d4b-b5d1-97f26887905c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:56.967864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84be14b-4058-4d4b-b5d1-97f26887905c admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.968131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84be14b-4058-4d4b-b5d1-97f26887905c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.968402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84be14b-4058-4d4b-b5d1-97f26887905c admin admin] PATH_INFO: `/v3/projects/7331ebb42734426ca789caed28181048/users/7b56509f76b1452eabdc54b093cdc2f0/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.979110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60441434-a4f7-490f-8d1e-2bc004b3ba14 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:56.983407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5711e5-4aec-4b47-9d06-11726892f971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:56.984056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5711e5-4aec-4b47-9d06-11726892f971 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:56.984271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84be14b-4058-4d4b-b5d1-97f26887905c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7331ebb42734426ca789caed28181048, user_id=7b56509f76b1452eabdc54b093cdc2f0, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:56.984370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5711e5-4aec-4b47-9d06-11726892f971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:56.985285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5711e5-4aec-4b47-9d06-11726892f971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:56.987133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84be14b-4058-4d4b-b5d1-97f26887905c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:56.994283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5711e5-4aec-4b47-9d06-11726892f971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:56.995711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5711e5-4aec-4b47-9d06-11726892f971 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.005041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 927/1871] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:25:57.059035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 945/1872] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:56 2022] PUT /identity/v3/projects/7331ebb42734426ca789caed28181048/users/7b56509f76b1452eabdc54b093cdc2f0/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:57.070033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f2fea-e25b-41aa-be2e-8f38696cbb38 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.070329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f2fea-e25b-41aa-be2e-8f38696cbb38 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.070870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f2fea-e25b-41aa-be2e-8f38696cbb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.099172 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-100f2fea-e25b-41aa-be2e-8f38696cbb38 None None] MFA Rules not processed for user `7b56509f76b1452eabdc54b093cdc2f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:57.100408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-100f2fea-e25b-41aa-be2e-8f38696cbb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:57.166140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 928/1873] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:25:57.171132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84be14b-4058-4d4b-b5d1-97f26887905c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:57.238215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84be14b-4058-4d4b-b5d1-97f26887905c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:57.247057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:57.247754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.248492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.248492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.248995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:57.250987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.384075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 946/1874] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:57 2022] POST /identity/v3/projects => generated 389 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:57.389183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f2fea-e25b-41aa-be2e-8f38696cbb38 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:57.420380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f2fea-e25b-41aa-be2e-8f38696cbb38 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:57.461412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caae903-ed0b-48dc-8a72-8727409a9dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:57.462745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caae903-ed0b-48dc-8a72-8727409a9dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.463111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caae903-ed0b-48dc-8a72-8727409a9dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.463713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caae903-ed0b-48dc-8a72-8727409a9dcd admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.464426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caae903-ed0b-48dc-8a72-8727409a9dcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:57.467058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caae903-ed0b-48dc-8a72-8727409a9dcd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.486205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:57.488075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:57.502534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbbe7f-93d3-4727-83b2-e18608d07699 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:57.508103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:57.508985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.509287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.509556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] PATH_INFO: `/v3/projects/4158957ed8a8403b943babd5fd3d3dee` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.510814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4158957ed8a8403b943babd5fd3d3dee)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:57.521508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.582254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 929/1875] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:57 2022] POST /identity/v3/users => generated 301 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:25:57.587432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caae903-ed0b-48dc-8a72-8727409a9dcd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:57.597111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caae903-ed0b-48dc-8a72-8727409a9dcd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:57.601702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2d729-57d5-4637-9c58-25c120315d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:57.602382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2d729-57d5-4637-9c58-25c120315d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.602704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2d729-57d5-4637-9c58-25c120315d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.602979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2d729-57d5-4637-9c58-25c120315d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.603729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2d729-57d5-4637-9c58-25c120315d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:57.605344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2d729-57d5-4637-9c58-25c120315d02 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.615171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 930/1876] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:25:57.619305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2d729-57d5-4637-9c58-25c120315d02 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:57.621499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4158957ed8a8403b943babd5fd3d3dee', 'request_id': 'req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:57.626688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4158957ed8a8403b943babd5fd3d3dee', 'request_id': 'req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:57.629728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2d729-57d5-4637-9c58-25c120315d02 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:57.694434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:57.695375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.695919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.696196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] PATH_INFO: `/v3/projects/e4a98f8dbcef419b8d10ab8547884273/users/6a255cd36862493a83f33c82b892d269/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.713591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a98f8dbcef419b8d10ab8547884273, user_id=6a255cd36862493a83f33c82b892d269, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:57.716391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.741831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4158957ed8a8403b943babd5fd3d3dee', 'request_id': 'req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:57.754909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] The token cache is being invalidate because project 4158957ed8a8403b943babd5fd3d3dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:57.755195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:57.755854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:57.757399 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 947/1877] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:57 2022] DELETE /identity/v3/projects/4158957ed8a8403b943babd5fd3d3dee => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:57.761183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:57.763300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:57.775883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9e47-4f6a-4ae1-94da-1c63615b6f7c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:57.780732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:57.781602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.781920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.782196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] PATH_INFO: `/v3/users/7b11b6d68be342518e1f1279720bd448` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.783533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b11b6d68be342518e1f1279720bd448)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:57.785447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.811505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 931/1878] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:57 2022] PUT /identity/v3/projects/e4a98f8dbcef419b8d10ab8547884273/users/6a255cd36862493a83f33c82b892d269/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:57.815552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:57.817213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:57.880398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb8d98-c912-43bf-bad7-c8bcdcc5133c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:57.885175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895b112-3f7e-48fb-afe3-78f3b83f9246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:57.885876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895b112-3f7e-48fb-afe3-78f3b83f9246 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.886162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895b112-3f7e-48fb-afe3-78f3b83f9246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.886466 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895b112-3f7e-48fb-afe3-78f3b83f9246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.887189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895b112-3f7e-48fb-afe3-78f3b83f9246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:57.888789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895b112-3f7e-48fb-afe3-78f3b83f9246 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.898464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 932/1879] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:57.902485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895b112-3f7e-48fb-afe3-78f3b83f9246 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:57.911677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895b112-3f7e-48fb-afe3-78f3b83f9246 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:57.916188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef604e7-e8a6-44fe-8aa0-3d2fd1e45b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:57.917011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef604e7-e8a6-44fe-8aa0-3d2fd1e45b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.917297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef604e7-e8a6-44fe-8aa0-3d2fd1e45b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.917574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef604e7-e8a6-44fe-8aa0-3d2fd1e45b70 admin admin] PATH_INFO: `/v3/projects/e4a98f8dbcef419b8d10ab8547884273/users/6a255cd36862493a83f33c82b892d269/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.920074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef604e7-e8a6-44fe-8aa0-3d2fd1e45b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a98f8dbcef419b8d10ab8547884273, user_id=6a255cd36862493a83f33c82b892d269, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:57.922695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef604e7-e8a6-44fe-8aa0-3d2fd1e45b70 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:57.932649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 933/1880] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:57 2022] PUT /identity/v3/projects/e4a98f8dbcef419b8d10ab8547884273/users/6a255cd36862493a83f33c82b892d269/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:57.943338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5b54-76f3-41b9-bb08-5575af61648b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:57.943567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5b54-76f3-41b9-bb08-5575af61648b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:57.943849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5b54-76f3-41b9-bb08-5575af61648b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:57.970974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1b1f5b54-76f3-41b9-bb08-5575af61648b None None] MFA Rules not processed for user `6a255cd36862493a83f33c82b892d269`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:57.972199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1b1f5b54-76f3-41b9-bb08-5575af61648b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:57.996952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b11b6d68be342518e1f1279720bd448'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:57.997657 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b11b6d68be342518e1f1279720bd448'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.038981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 934/1881] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:58.050174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5b54-76f3-41b9-bb08-5575af61648b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:58.051516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1b1f5b54-76f3-41b9-bb08-5575af61648b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:58.082369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b11b6d68be342518e1f1279720bd448'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.089760 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b11b6d68be342518e1f1279720bd448'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.090441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b11b6d68be342518e1f1279720bd448'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.107247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5b54-76f3-41b9-bb08-5575af61648b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:58.107769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 948/1882] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:57 2022] DELETE /identity/v3/users/7b11b6d68be342518e1f1279720bd448 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:58.156595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:58.157250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:58.157539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:58.157845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:58.219919 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:58.221363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:58.231901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 935/1883] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:58.363516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:58.364997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:58.445665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ee036-6911-4098-887c-2ab81d7f73a0 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:58.452147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:58.453039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:58.453325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:58.453604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] PATH_INFO: `/v3/users/7b56509f76b1452eabdc54b093cdc2f0` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:58.455089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b56509f76b1452eabdc54b093cdc2f0)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:58.457212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:58.632937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b56509f76b1452eabdc54b093cdc2f0'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.633603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b56509f76b1452eabdc54b093cdc2f0'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.678910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:58.680350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:58.694198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eff70a-fc89-4d87-81b5-fb1ff90798a8 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:58.699683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:58.700700 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:58.700988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:58.701264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] PATH_INFO: `/v3/projects/806cc8227c184c01afa55ea1d2b61236` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:58.703343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] RBAC: Authorizing `identity:delete_project(project_id=806cc8227c184c01afa55ea1d2b61236)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:58.708064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:58.724010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b56509f76b1452eabdc54b093cdc2f0'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.739429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b56509f76b1452eabdc54b093cdc2f0'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.743315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8ac632a4-4bd7-4d9f-a365-dc7c2b2a134f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b56509f76b1452eabdc54b093cdc2f0'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.766712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 949/1884] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:58 2022] DELETE /identity/v3/users/7b56509f76b1452eabdc54b093cdc2f0 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:58.770939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec7e1-67a3-4a07-950a-8d0d98663112 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:58.771234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec7e1-67a3-4a07-950a-8d0d98663112 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:58.771538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec7e1-67a3-4a07-950a-8d0d98663112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:58.789457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-671ec7e1-67a3-4a07-950a-8d0d98663112 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:58.790722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-671ec7e1-67a3-4a07-950a-8d0d98663112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:58.834888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '806cc8227c184c01afa55ea1d2b61236', 'request_id': 'req-d90007db-8001-4b7e-9d25-2ecde862864d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.840717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '806cc8227c184c01afa55ea1d2b61236', 'request_id': 'req-d90007db-8001-4b7e-9d25-2ecde862864d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.849997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 950/1885] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:58.860360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec7e1-67a3-4a07-950a-8d0d98663112 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:58.861687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-671ec7e1-67a3-4a07-950a-8d0d98663112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:58.915671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec7e1-67a3-4a07-950a-8d0d98663112 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:58.918372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '806cc8227c184c01afa55ea1d2b61236', 'request_id': 'req-d90007db-8001-4b7e-9d25-2ecde862864d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.929976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] The token cache is being invalidate because project 806cc8227c184c01afa55ea1d2b61236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:58.930264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.931045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d90007db-8001-4b7e-9d25-2ecde862864d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:58.932844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 936/1886] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:58 2022] DELETE /identity/v3/projects/806cc8227c184c01afa55ea1d2b61236 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:58.937482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2216b-c19e-4a28-bd0d-28f0e1005af2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:58.937826 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2216b-c19e-4a28-bd0d-28f0e1005af2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:58.938174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2216b-c19e-4a28-bd0d-28f0e1005af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:58.956330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-50e2216b-c19e-4a28-bd0d-28f0e1005af2 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:58.957521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-50e2216b-c19e-4a28-bd0d-28f0e1005af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:58.963080 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:58.963676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:58.963966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:58.964243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:58.966706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:59.016393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 937/1887] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:59.024917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf19bf-d9a5-43f3-918b-fd824f9fae25 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.025209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf19bf-d9a5-43f3-918b-fd824f9fae25 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.025491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf19bf-d9a5-43f3-918b-fd824f9fae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.029982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.031433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.041016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 951/1888] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:59.042906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5cbf19bf-d9a5-43f3-918b-fd824f9fae25 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:59.044134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5cbf19bf-d9a5-43f3-918b-fd824f9fae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:59.052476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 938/1889] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:59.056622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:59.066417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd4773-f3bc-456f-83bc-f53394ba8789 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:59.071209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:59.071864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.072149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.072421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.073195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:25:59.073500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.075711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.087490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 952/1890] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:25:59.091989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf19bf-d9a5-43f3-918b-fd824f9fae25 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:59.101368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf19bf-d9a5-43f3-918b-fd824f9fae25 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:59.106215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241089d-3369-4d70-bf51-ed3d13bfc6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:59.106879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241089d-3369-4d70-bf51-ed3d13bfc6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.107161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241089d-3369-4d70-bf51-ed3d13bfc6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.107437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241089d-3369-4d70-bf51-ed3d13bfc6e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.108154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241089d-3369-4d70-bf51-ed3d13bfc6e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.110105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241089d-3369-4d70-bf51-ed3d13bfc6e0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.205259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 939/1891] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:25:59 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:59.209362 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:59.219592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2166f-abf9-46cb-9c18-c443f710ceaf admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:59.275858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7b30-cea3-4b62-bfa6-5df3154c4a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:59.276543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7b30-cea3-4b62-bfa6-5df3154c4a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.276806 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7b30-cea3-4b62-bfa6-5df3154c4a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.277125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7b30-cea3-4b62-bfa6-5df3154c4a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.277907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df7b30-cea3-4b62-bfa6-5df3154c4a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.279880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df7b30-cea3-4b62-bfa6-5df3154c4a5f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.419331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 953/1892] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:25:59 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:25:59.420533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241089d-3369-4d70-bf51-ed3d13bfc6e0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:59.430690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241089d-3369-4d70-bf51-ed3d13bfc6e0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:59.438935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca49ac-a944-4c45-9033-d3479b7e58ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:59.438935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca49ac-a944-4c45-9033-d3479b7e58ac admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.438935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca49ac-a944-4c45-9033-d3479b7e58ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.438935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca49ac-a944-4c45-9033-d3479b7e58ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.439318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca49ac-a944-4c45-9033-d3479b7e58ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.439468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca49ac-a944-4c45-9033-d3479b7e58ac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.450498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 940/1893] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:59.454489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7b30-cea3-4b62-bfa6-5df3154c4a5f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:59.464330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7b30-cea3-4b62-bfa6-5df3154c4a5f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:59.468981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:59.469714 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.469958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.470236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.470992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.472615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.483484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 954/1894] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:25:59.487369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca49ac-a944-4c45-9033-d3479b7e58ac admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:59.499696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca49ac-a944-4c45-9033-d3479b7e58ac admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:59.504127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bdbb3-e7f4-428e-8e3d-773c1b6f3832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:59.505125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bdbb3-e7f4-428e-8e3d-773c1b6f3832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.505558 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bdbb3-e7f4-428e-8e3d-773c1b6f3832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.510212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bdbb3-e7f4-428e-8e3d-773c1b6f3832 admin admin] PATH_INFO: `/v3/projects/6dc12e51fbc2450ab2d7868463bff90e/users/fe466c6e49404e52b44cc49ef1475140/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.514213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:59.516291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:59.538535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf4cd7-5f0e-42ab-9262-8c04d2b71984 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:59.547268 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:59.550626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bdbb3-e7f4-428e-8e3d-773c1b6f3832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc12e51fbc2450ab2d7868463bff90e, user_id=fe466c6e49404e52b44cc49ef1475140, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.553328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bdbb3-e7f4-428e-8e3d-773c1b6f3832 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.559544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.560341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.560815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] PATH_INFO: `/v3/projects/7331ebb42734426ca789caed28181048` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.563229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7331ebb42734426ca789caed28181048)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.565453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.644346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 941/1895] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:25:59 2022] PUT /identity/v3/projects/6dc12e51fbc2450ab2d7868463bff90e/users/fe466c6e49404e52b44cc49ef1475140/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:25:59.649524 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388d8ee-198d-492d-9057-1980d5990146 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.649853 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388d8ee-198d-492d-9057-1980d5990146 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.650140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388d8ee-198d-492d-9057-1980d5990146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.667073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-2388d8ee-198d-492d-9057-1980d5990146 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:25:59.668801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2388d8ee-198d-492d-9057-1980d5990146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:59.728533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 942/1896] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:25:59.738872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388d8ee-198d-492d-9057-1980d5990146 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:25:59.743830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2388d8ee-198d-492d-9057-1980d5990146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:25:59.795340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7331ebb42734426ca789caed28181048', 'request_id': 'req-09d5f293-e5b9-41e5-a0b0-483076eba262'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:59.810375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7331ebb42734426ca789caed28181048', 'request_id': 'req-09d5f293-e5b9-41e5-a0b0-483076eba262'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:59.833243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388d8ee-198d-492d-9057-1980d5990146 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:25:59.884820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8e17c-e523-4a75-a688-7c3788bd3a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:25:59.885477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8e17c-e523-4a75-a688-7c3788bd3a46 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:25:59.885831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8e17c-e523-4a75-a688-7c3788bd3a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:25:59.886179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8e17c-e523-4a75-a688-7c3788bd3a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:25:59.947555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8e17c-e523-4a75-a688-7c3788bd3a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:25:59.949028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8e17c-e523-4a75-a688-7c3788bd3a46 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:25:59.958980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 943/1897] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:25:59.970142 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7331ebb42734426ca789caed28181048', 'request_id': 'req-09d5f293-e5b9-41e5-a0b0-483076eba262'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:59.983244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] The token cache is being invalidate because project 7331ebb42734426ca789caed28181048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:25:59.983607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:59.984326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:25:59.986003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 955/1898] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:25:59 2022] DELETE /identity/v3/projects/7331ebb42734426ca789caed28181048 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:25:59.991903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8e17c-e523-4a75-a688-7c3788bd3a46 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:00.007477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8e17c-e523-4a75-a688-7c3788bd3a46 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:00.014780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:00.016178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:00.016638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:00.017109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] PATH_INFO: `/v3/users/6a255cd36862493a83f33c82b892d269` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:00.019121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a255cd36862493a83f33c82b892d269)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:00.021363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:00.208881 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a255cd36862493a83f33c82b892d269'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:00.209559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a255cd36862493a83f33c82b892d269'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:00.210198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a255cd36862493a83f33c82b892d269'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:00.217722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a255cd36862493a83f33c82b892d269'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:00.336297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a255cd36862493a83f33c82b892d269'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:00.359648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 944/1899] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:25:59 2022] DELETE /identity/v3/users/6a255cd36862493a83f33c82b892d269 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:00.818733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:00.820391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:00.858895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5f293-e5b9-41e5-a0b0-483076eba262 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:01.041962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:01.043595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.045153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.045532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] PATH_INFO: `/v3/projects/e4a98f8dbcef419b8d10ab8547884273` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.054736 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a98f8dbcef419b8d10ab8547884273)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:01.063110 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:01.229888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a98f8dbcef419b8d10ab8547884273', 'request_id': 'req-513cc0a9-2281-4816-9c1c-7ae112402967'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.231118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:01.233101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:01.237227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a98f8dbcef419b8d10ab8547884273', 'request_id': 'req-513cc0a9-2281-4816-9c1c-7ae112402967'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.246953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04860a2-ab71-42d0-a509-1f6b9dac796c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:01.313189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a98f8dbcef419b8d10ab8547884273', 'request_id': 'req-513cc0a9-2281-4816-9c1c-7ae112402967'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.330223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:01.331999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.332599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.333244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] PATH_INFO: `/v3/users/bac2de2db4394a5193f9ad9a3c02816b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.339510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] The token cache is being invalidate because project e4a98f8dbcef419b8d10ab8547884273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:01.341127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bac2de2db4394a5193f9ad9a3c02816b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:01.343170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:01.343583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.346927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-513cc0a9-2281-4816-9c1c-7ae112402967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.357207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 956/1900] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:00 2022] DELETE /identity/v3/projects/e4a98f8dbcef419b8d10ab8547884273 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:01.363055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b07574-107e-4aad-8d5f-6ccc448bce66 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.363803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b07574-107e-4aad-8d5f-6ccc448bce66 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.364695 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b07574-107e-4aad-8d5f-6ccc448bce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.395746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f4b07574-107e-4aad-8d5f-6ccc448bce66 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:01.395746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f4b07574-107e-4aad-8d5f-6ccc448bce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:01.399296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 957/1901] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:01.407236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b3a4-bbe0-41fb-975b-5a2a8dda3b92 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.407513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b3a4-bbe0-41fb-975b-5a2a8dda3b92 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.407803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b3a4-bbe0-41fb-975b-5a2a8dda3b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.437765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-dc26b3a4-bbe0-41fb-975b-5a2a8dda3b92 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:01.437765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-dc26b3a4-bbe0-41fb-975b-5a2a8dda3b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:01.441838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 958/1902] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:01.447267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26b3a4-bbe0-41fb-975b-5a2a8dda3b92 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:01.466431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26b3a4-bbe0-41fb-975b-5a2a8dda3b92 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:01.479975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:01.481417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.481815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.482163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.482961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:01.483334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:01.485379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:01.521365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 959/1903] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:01.522568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:01.537127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2192ae-31c7-42c9-b53e-11b90871dc35 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:01.543407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:01.544168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.544550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.545135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.546117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:01.548727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:01.611330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bac2de2db4394a5193f9ad9a3c02816b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.618531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bac2de2db4394a5193f9ad9a3c02816b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.619634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bac2de2db4394a5193f9ad9a3c02816b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.634908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bac2de2db4394a5193f9ad9a3c02816b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.690277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 960/1904] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:01 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:01.694403 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:01.702044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:01.703037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-74ce3fa9-ca64-42ae-8676-444bd4af22d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bac2de2db4394a5193f9ad9a3c02816b'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:01.738738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 945/1905] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:01 2022] DELETE /identity/v3/users/bac2de2db4394a5193f9ad9a3c02816b => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:01.743211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3215-a81f-4082-bb10-daa35736fe80 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.743660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3215-a81f-4082-bb10-daa35736fe80 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.744003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3215-a81f-4082-bb10-daa35736fe80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.778862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9a7e3215-a81f-4082-bb10-daa35736fe80 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:01.779956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9a7e3215-a81f-4082-bb10-daa35736fe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:01.866189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec329ba-0e27-46ec-9e0d-90cb4fadeed6 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:01.945500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 946/1906] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:01.947505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db818ce-5942-4ab9-a8fa-03c7d791e8f5 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.948433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db818ce-5942-4ab9-a8fa-03c7d791e8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.948433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db818ce-5942-4ab9-a8fa-03c7d791e8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.966560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:01.967541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:01.969577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:01.969577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:01.971893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:01.974104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:01.998973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5db818ce-5942-4ab9-a8fa-03c7d791e8f5 None None] MFA Rules not processed for user `fe466c6e49404e52b44cc49ef1475140`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:02.000312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5db818ce-5942-4ab9-a8fa-03c7d791e8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:02.076065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 947/1907] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:02.076065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db818ce-5942-4ab9-a8fa-03c7d791e8f5 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.076065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5db818ce-5942-4ab9-a8fa-03c7d791e8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:02.083118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 961/1908] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:01 2022] POST /identity/v3/users => generated 290 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:02.086739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.088674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:02.101701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a10e7b-84f4-4b79-9b35-2a0a174e96f2 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.109266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4303a-3b2f-41d1-8b74-19363bd4fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.109990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4303a-3b2f-41d1-8b74-19363bd4fcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.110332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4303a-3b2f-41d1-8b74-19363bd4fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.110612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4303a-3b2f-41d1-8b74-19363bd4fcbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.111312 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4303a-3b2f-41d1-8b74-19363bd4fcbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.113677 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4303a-3b2f-41d1-8b74-19363bd4fcbe admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.137329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db818ce-5942-4ab9-a8fa-03c7d791e8f5 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.141319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4ee75-7005-4009-ad92-cd6482a3d56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.142022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4ee75-7005-4009-ad92-cd6482a3d56e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.142312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4ee75-7005-4009-ad92-cd6482a3d56e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.142617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4ee75-7005-4009-ad92-cd6482a3d56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.152756 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4ee75-7005-4009-ad92-cd6482a3d56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.154253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4ee75-7005-4009-ad92-cd6482a3d56e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.164612 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 948/1909] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:02.171338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4ee75-7005-4009-ad92-cd6482a3d56e service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.188278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4ee75-7005-4009-ad92-cd6482a3d56e service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.194685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979e3d9-b1b6-434d-944a-7c66f2fbe088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.195509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e3d9-b1b6-434d-944a-7c66f2fbe088 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.196068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e3d9-b1b6-434d-944a-7c66f2fbe088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.196345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e3d9-b1b6-434d-944a-7c66f2fbe088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.197102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979e3d9-b1b6-434d-944a-7c66f2fbe088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.199965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979e3d9-b1b6-434d-944a-7c66f2fbe088 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.212646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 962/1910] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:02 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:02.214041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 949/1911] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:02.216323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979e3d9-b1b6-434d-944a-7c66f2fbe088 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.217369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4303a-3b2f-41d1-8b74-19363bd4fcbe admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.229888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979e3d9-b1b6-434d-944a-7c66f2fbe088 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.310933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4303a-3b2f-41d1-8b74-19363bd4fcbe admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.314364 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc08c3d-412b-48fe-86c2-39d319b6f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.315068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08c3d-412b-48fe-86c2-39d319b6f163 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.315357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08c3d-412b-48fe-86c2-39d319b6f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.315640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08c3d-412b-48fe-86c2-39d319b6f163 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.316354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc08c3d-412b-48fe-86c2-39d319b6f163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.317717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54b7e4-b51e-4ca7-a691-906cdb1e21f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.319154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc08c3d-412b-48fe-86c2-39d319b6f163 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.319438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54b7e4-b51e-4ca7-a691-906cdb1e21f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.320001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54b7e4-b51e-4ca7-a691-906cdb1e21f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.320989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54b7e4-b51e-4ca7-a691-906cdb1e21f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.322308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54b7e4-b51e-4ca7-a691-906cdb1e21f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.325327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54b7e4-b51e-4ca7-a691-906cdb1e21f9 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.346013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 963/1912] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:02.347206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54b7e4-b51e-4ca7-a691-906cdb1e21f9 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.359688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54b7e4-b51e-4ca7-a691-906cdb1e21f9 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.366102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ebc7d-7103-42b2-a920-095fd7d366cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.367009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ebc7d-7103-42b2-a920-095fd7d366cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.367304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ebc7d-7103-42b2-a920-095fd7d366cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.367575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ebc7d-7103-42b2-a920-095fd7d366cc admin admin] PATH_INFO: `/v3/projects/248d93ad559f416893e1a8d97ab0a45f/users/4601bfe2b69b4b2fb268a593bebbe2a7/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.388260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ebc7d-7103-42b2-a920-095fd7d366cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=248d93ad559f416893e1a8d97ab0a45f, user_id=4601bfe2b69b4b2fb268a593bebbe2a7, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.391383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ebc7d-7103-42b2-a920-095fd7d366cc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.483904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 950/1913] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:02 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:02.485018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc08c3d-412b-48fe-86c2-39d319b6f163 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.496824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc08c3d-412b-48fe-86c2-39d319b6f163 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.503064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f728580-13d7-4185-96b6-7bb98a192609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.504538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f728580-13d7-4185-96b6-7bb98a192609 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.504538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f728580-13d7-4185-96b6-7bb98a192609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.504538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f728580-13d7-4185-96b6-7bb98a192609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.505098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f728580-13d7-4185-96b6-7bb98a192609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.506951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f728580-13d7-4185-96b6-7bb98a192609 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.519390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 951/1914] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:02.523563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f728580-13d7-4185-96b6-7bb98a192609 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.534255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f728580-13d7-4185-96b6-7bb98a192609 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.545997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 964/1915] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:02 2022] PUT /identity/v3/projects/248d93ad559f416893e1a8d97ab0a45f/users/4601bfe2b69b4b2fb268a593bebbe2a7/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:02.549495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155e647-855e-4dfd-81b2-e70255a81a2d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.550088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155e647-855e-4dfd-81b2-e70255a81a2d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.550375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155e647-855e-4dfd-81b2-e70255a81a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.581572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1155e647-855e-4dfd-81b2-e70255a81a2d None None] MFA Rules not processed for user `4601bfe2b69b4b2fb268a593bebbe2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:02.586131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1155e647-855e-4dfd-81b2-e70255a81a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:02.596451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de534988-a8e7-4f70-b924-3e70168e60ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.597459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534988-a8e7-4f70-b924-3e70168e60ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.597751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534988-a8e7-4f70-b924-3e70168e60ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.598068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534988-a8e7-4f70-b924-3e70168e60ce admin admin] PATH_INFO: `/v3/projects/6183251fc8a54a829b220a38088df86d/users/30c0f366d1e445ad91d56b1725d3b093/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.615556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de534988-a8e7-4f70-b924-3e70168e60ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6183251fc8a54a829b220a38088df86d, user_id=30c0f366d1e445ad91d56b1725d3b093, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.619395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de534988-a8e7-4f70-b924-3e70168e60ce admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.661331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 965/1916] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:02.665890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155e647-855e-4dfd-81b2-e70255a81a2d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.677056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155e647-855e-4dfd-81b2-e70255a81a2d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.682649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c23eb-c6ac-4f2f-abda-76cba26f48f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.684229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c23eb-c6ac-4f2f-abda-76cba26f48f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.684569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c23eb-c6ac-4f2f-abda-76cba26f48f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.684851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c23eb-c6ac-4f2f-abda-76cba26f48f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.685962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c23eb-c6ac-4f2f-abda-76cba26f48f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.687926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c23eb-c6ac-4f2f-abda-76cba26f48f3 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.709696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 952/1917] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:02 2022] PUT /identity/v3/projects/6183251fc8a54a829b220a38088df86d/users/30c0f366d1e445ad91d56b1725d3b093/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:02.710837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de534988-a8e7-4f70-b924-3e70168e60ce admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.778417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de534988-a8e7-4f70-b924-3e70168e60ce admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.790335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e580af-6615-4fdb-9452-cf334889e7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.791209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e580af-6615-4fdb-9452-cf334889e7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.791496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e580af-6615-4fdb-9452-cf334889e7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.791823 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e580af-6615-4fdb-9452-cf334889e7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.792563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e580af-6615-4fdb-9452-cf334889e7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.794884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e580af-6615-4fdb-9452-cf334889e7f0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.813850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 953/1918] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:02.819251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e580af-6615-4fdb-9452-cf334889e7f0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.834414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e580af-6615-4fdb-9452-cf334889e7f0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:02.840209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad7aea-a983-4346-82bf-ded030e443b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:02.845390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7aea-a983-4346-82bf-ded030e443b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:02.845682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7aea-a983-4346-82bf-ded030e443b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:02.845993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7aea-a983-4346-82bf-ded030e443b8 admin admin] PATH_INFO: `/v3/projects/6183251fc8a54a829b220a38088df86d/users/30c0f366d1e445ad91d56b1725d3b093/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:02.849006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad7aea-a983-4346-82bf-ded030e443b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6183251fc8a54a829b220a38088df86d, user_id=30c0f366d1e445ad91d56b1725d3b093, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:02.851191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad7aea-a983-4346-82bf-ded030e443b8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:02.867161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 954/1919] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:02 2022] PUT /identity/v3/projects/6183251fc8a54a829b220a38088df86d/users/30c0f366d1e445ad91d56b1725d3b093/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:02.890656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 966/1920] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:02 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:02.892544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad7aea-a983-4346-82bf-ded030e443b8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:02.963061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad7aea-a983-4346-82bf-ded030e443b8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:03.025582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:03.026563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.027638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.027918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.028634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:03.030730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:03.107724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 955/1921] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:02 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:03.111447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c23eb-c6ac-4f2f-abda-76cba26f48f3 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:03.117307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:03.119452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:03.121981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c23eb-c6ac-4f2f-abda-76cba26f48f3 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:03.126715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c8105-e2c0-4460-a02f-9094a6cab9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:03.127573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c8105-e2c0-4460-a02f-9094a6cab9fe admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.127950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c8105-e2c0-4460-a02f-9094a6cab9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.128381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c8105-e2c0-4460-a02f-9094a6cab9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.129172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c8105-e2c0-4460-a02f-9094a6cab9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:03.131027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c8105-e2c0-4460-a02f-9094a6cab9fe admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:03.132846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdd20c-4199-4d06-ab20-3c740be8b104 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:03.138158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:03.139059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.141060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.141060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] PATH_INFO: `/v3/projects/ba1e3fb2ab0443e6bbe2f304a60cd709` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.141060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba1e3fb2ab0443e6bbe2f304a60cd709)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:03.142715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 967/1922] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:03.143245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:03.146628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c8105-e2c0-4460-a02f-9094a6cab9fe admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:03.159849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c8105-e2c0-4460-a02f-9094a6cab9fe admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:03.168579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450978f1-03d8-4a6c-b170-71571d1274ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:03.169628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450978f1-03d8-4a6c-b170-71571d1274ac admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.171221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450978f1-03d8-4a6c-b170-71571d1274ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.171648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450978f1-03d8-4a6c-b170-71571d1274ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.186631 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450978f1-03d8-4a6c-b170-71571d1274ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:03.199586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450978f1-03d8-4a6c-b170-71571d1274ac admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:03.215593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 968/1923] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:03.221598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450978f1-03d8-4a6c-b170-71571d1274ac admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:03.235659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450978f1-03d8-4a6c-b170-71571d1274ac admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:03.240828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582385f9-3b59-4789-bbe3-6922ccd1eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:03.241720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582385f9-3b59-4789-bbe3-6922ccd1eb26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.242039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582385f9-3b59-4789-bbe3-6922ccd1eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.242325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582385f9-3b59-4789-bbe3-6922ccd1eb26 admin admin] PATH_INFO: `/v3/projects/48bf5d49bdd440bca669489e9d26d9c0/users/9a1094ec56f7413989f87af30f1f2e63/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.267496 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582385f9-3b59-4789-bbe3-6922ccd1eb26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bf5d49bdd440bca669489e9d26d9c0, user_id=9a1094ec56f7413989f87af30f1f2e63, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:03.270352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582385f9-3b59-4789-bbe3-6922ccd1eb26 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:03.278902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1e3fb2ab0443e6bbe2f304a60cd709', 'request_id': 'req-39308f8d-85e7-48df-9bf6-40e95e4fa930'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:03.285171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1e3fb2ab0443e6bbe2f304a60cd709', 'request_id': 'req-39308f8d-85e7-48df-9bf6-40e95e4fa930'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:03.365019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 969/1924] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:03 2022] PUT /identity/v3/projects/48bf5d49bdd440bca669489e9d26d9c0/users/9a1094ec56f7413989f87af30f1f2e63/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:03.369426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6f1f-3fa4-495b-961f-a4ca6813a0f2 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.369835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6f1f-3fa4-495b-961f-a4ca6813a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.371064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb6f1f-3fa4-495b-961f-a4ca6813a0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.399014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1e3fb2ab0443e6bbe2f304a60cd709', 'request_id': 'req-39308f8d-85e7-48df-9bf6-40e95e4fa930'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:03.416720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] The token cache is being invalidate because project ba1e3fb2ab0443e6bbe2f304a60cd709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:03.417058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:03.417723 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:03.420784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-08cb6f1f-3fa4-495b-961f-a4ca6813a0f2 None None] MFA Rules not processed for user `9a1094ec56f7413989f87af30f1f2e63`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:03.422589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-08cb6f1f-3fa4-495b-961f-a4ca6813a0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:03.432994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 956/1925] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:03 2022] DELETE /identity/v3/projects/ba1e3fb2ab0443e6bbe2f304a60cd709 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:03.436751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:03.439022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:03.594755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39308f8d-85e7-48df-9bf6-40e95e4fa930 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:03.612288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 970/1926] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:03.620536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb6f1f-3fa4-495b-961f-a4ca6813a0f2 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:03.621499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-08cb6f1f-3fa4-495b-961f-a4ca6813a0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:03.684398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:03.685554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.686050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.686501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] PATH_INFO: `/v3/users/5bc73b03f26a426cbdf7a6b8ccc08d86` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.687912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc73b03f26a426cbdf7a6b8ccc08d86)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:03.705880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:03.707574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb6f1f-3fa4-495b-961f-a4ca6813a0f2 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:03.720597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bdad4-b5c2-439f-85c9-cc27cda29b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:03.721289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bdad4-b5c2-439f-85c9-cc27cda29b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.721576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bdad4-b5c2-439f-85c9-cc27cda29b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.721887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bdad4-b5c2-439f-85c9-cc27cda29b5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.722649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bdad4-b5c2-439f-85c9-cc27cda29b5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:03.724603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bdad4-b5c2-439f-85c9-cc27cda29b5c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:03.899522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 971/1927] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:03 2022] POST /identity/v3/projects => generated 377 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:03.904260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bdad4-b5c2-439f-85c9-cc27cda29b5c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:03.967609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bdad4-b5c2-439f-85c9-cc27cda29b5c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:03.973276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:03.974251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:03.974543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:03.977097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:03.977097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:03.977503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:04.022010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc73b03f26a426cbdf7a6b8ccc08d86'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:04.022731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc73b03f26a426cbdf7a6b8ccc08d86'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:04.023416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc73b03f26a426cbdf7a6b8ccc08d86'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:04.031713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc73b03f26a426cbdf7a6b8ccc08d86'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:04.130416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 972/1928] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:03 2022] POST /identity/v3/users => generated 295 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:04.134735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:04.137310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:04.141526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc73b03f26a426cbdf7a6b8ccc08d86'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:04.161811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d7ccd-34b0-40e1-a22c-1ae11aef1961 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:04.169005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248fa23-516f-4dea-a76d-a33e6067ed2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:04.170019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248fa23-516f-4dea-a76d-a33e6067ed2a admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:04.170494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248fa23-516f-4dea-a76d-a33e6067ed2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:04.170956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248fa23-516f-4dea-a76d-a33e6067ed2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:04.172186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248fa23-516f-4dea-a76d-a33e6067ed2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:04.172994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 957/1929] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:03 2022] DELETE /identity/v3/users/5bc73b03f26a426cbdf7a6b8ccc08d86 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:04.175313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248fa23-516f-4dea-a76d-a33e6067ed2a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:04.187618 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 973/1930] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:04.191958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:04.271225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e604e11-1fa9-444d-86e1-89ecbfdbb006 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:04.280217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416908c-e62e-4bc7-8004-0dc96d9077f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:04.281211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416908c-e62e-4bc7-8004-0dc96d9077f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:04.281502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416908c-e62e-4bc7-8004-0dc96d9077f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:04.281803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416908c-e62e-4bc7-8004-0dc96d9077f5 admin admin] PATH_INFO: `/v3/projects/d65bf918451540e9890385fa05d1b066/users/81fe76d3a3944648ad937eb9d41b59f7/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:04.298311 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416908c-e62e-4bc7-8004-0dc96d9077f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65bf918451540e9890385fa05d1b066, user_id=81fe76d3a3944648ad937eb9d41b59f7, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:04.301180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416908c-e62e-4bc7-8004-0dc96d9077f5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:04.411189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 958/1931] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:04 2022] PUT /identity/v3/projects/d65bf918451540e9890385fa05d1b066/users/81fe76d3a3944648ad937eb9d41b59f7/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:04.415580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248fa23-516f-4dea-a76d-a33e6067ed2a admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:04.430187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248fa23-516f-4dea-a76d-a33e6067ed2a admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:04.554640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1790a58-7034-445f-9f3c-da364fc28e95 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:04.554640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1790a58-7034-445f-9f3c-da364fc28e95 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:04.554640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1790a58-7034-445f-9f3c-da364fc28e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:04.554877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411f5a7-62b3-47ed-a145-1b7d4da4a56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:04.554877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411f5a7-62b3-47ed-a145-1b7d4da4a56d admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:04.554877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411f5a7-62b3-47ed-a145-1b7d4da4a56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:04.554877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411f5a7-62b3-47ed-a145-1b7d4da4a56d admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:04.555174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411f5a7-62b3-47ed-a145-1b7d4da4a56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:04.555174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411f5a7-62b3-47ed-a145-1b7d4da4a56d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:04.572650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 974/1932] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:04.583096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411f5a7-62b3-47ed-a145-1b7d4da4a56d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:04.597962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411f5a7-62b3-47ed-a145-1b7d4da4a56d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:04.604232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848a6c-380b-468d-b073-7af3300c539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:04.606395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848a6c-380b-468d-b073-7af3300c539b admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:04.606395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848a6c-380b-468d-b073-7af3300c539b admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:04.606395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848a6c-380b-468d-b073-7af3300c539b admin admin] PATH_INFO: `/v3/projects/d65bf918451540e9890385fa05d1b066/users/81fe76d3a3944648ad937eb9d41b59f7/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:04.609300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2848a6c-380b-468d-b073-7af3300c539b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65bf918451540e9890385fa05d1b066, user_id=81fe76d3a3944648ad937eb9d41b59f7, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:04.612551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2848a6c-380b-468d-b073-7af3300c539b admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:04.626263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 975/1933] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:04 2022] PUT /identity/v3/projects/d65bf918451540e9890385fa05d1b066/users/81fe76d3a3944648ad937eb9d41b59f7/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:04.636780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a289c-30b3-440d-bc2c-fbee9899692d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:04.637223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a289c-30b3-440d-bc2c-fbee9899692d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:04.637517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a289c-30b3-440d-bc2c-fbee9899692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:04.641119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f1790a58-7034-445f-9f3c-da364fc28e95 None None] MFA Rules not processed for user `30c0f366d1e445ad91d56b1725d3b093`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:04.642433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f1790a58-7034-445f-9f3c-da364fc28e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:04.669454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-4b7a289c-30b3-440d-bc2c-fbee9899692d None None] MFA Rules not processed for user `81fe76d3a3944648ad937eb9d41b59f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:04.670534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4b7a289c-30b3-440d-bc2c-fbee9899692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:04.784668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 959/1934] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:04.799376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1790a58-7034-445f-9f3c-da364fc28e95 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:04.801302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f1790a58-7034-445f-9f3c-da364fc28e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:04.835747 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 976/1935] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:04.847054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a289c-30b3-440d-bc2c-fbee9899692d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:04.847963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4b7a289c-30b3-440d-bc2c-fbee9899692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:04.903320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1790a58-7034-445f-9f3c-da364fc28e95 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:04.909175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:04.909886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:04.910238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:04.910445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:04.912279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:04.952287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a289c-30b3-440d-bc2c-fbee9899692d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:04.956906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:04.957283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:04.957571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:04.957893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:04.969833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:04.972137 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:04.998727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 977/1936] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:05.003163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:05.008548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:05.008715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:05.011345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:05.037336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 960/1937] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:05.122037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc91e0-60c7-42c1-ac42-70e4e4e30c26 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:05.130933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:05.132049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:05.133095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:05.133381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:05.133665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] PATH_INFO: `/v3/projects/1840529b8cda4a8ea0790e1f854c6263` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:05.135001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1840529b8cda4a8ea0790e1f854c6263)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:05.137516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:05.146365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02913271-4647-479f-9fd8-a55006a92bbe service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:05.152451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b3f55-fcfa-452d-9b3b-0a0bbbb7fe0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:05.153426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b3f55-fcfa-452d-9b3b-0a0bbbb7fe0a service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:05.153844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b3f55-fcfa-452d-9b3b-0a0bbbb7fe0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:05.154332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b3f55-fcfa-452d-9b3b-0a0bbbb7fe0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:05.157347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a99b3f55-fcfa-452d-9b3b-0a0bbbb7fe0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:05.270001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1840529b8cda4a8ea0790e1f854c6263', 'request_id': 'req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:05.277385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1840529b8cda4a8ea0790e1f854c6263', 'request_id': 'req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:05.286864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b3f55-fcfa-452d-9b3b-0a0bbbb7fe0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:05.288820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b3f55-fcfa-452d-9b3b-0a0bbbb7fe0a service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:05.305140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 961/1938] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:05.366491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1840529b8cda4a8ea0790e1f854c6263', 'request_id': 'req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:05.379452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] The token cache is being invalidate because project 1840529b8cda4a8ea0790e1f854c6263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:05.379755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:05.380418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3eb906eb-b6a1-45a6-bba1-5dcbc7682051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:05.382612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 978/1939] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:04 2022] DELETE /identity/v3/projects/1840529b8cda4a8ea0790e1f854c6263 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:05.387440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69ff13-379e-4412-a207-3654f1efdcc9 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:05.387940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69ff13-379e-4412-a207-3654f1efdcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:05.388375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69ff13-379e-4412-a207-3654f1efdcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:05.411416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fb69ff13-379e-4412-a207-3654f1efdcc9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:05.413400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fb69ff13-379e-4412-a207-3654f1efdcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:05.522106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 962/1940] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:05.529751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680edd92-20a9-4aff-adb5-3499f7db68de None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:05.529920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680edd92-20a9-4aff-adb5-3499f7db68de None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:05.530102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680edd92-20a9-4aff-adb5-3499f7db68de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:05.550328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-680edd92-20a9-4aff-adb5-3499f7db68de None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:05.552580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-680edd92-20a9-4aff-adb5-3499f7db68de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:05.568666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 979/1941] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:05.570096 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69ff13-379e-4412-a207-3654f1efdcc9 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:05.584338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69ff13-379e-4412-a207-3654f1efdcc9 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:05.598404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:05.599101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:05.599359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:05.599638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:05.600376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:05.600690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:05.606451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:05.621758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 963/1942] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:05.626124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680edd92-20a9-4aff-adb5-3499f7db68de None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:05.635213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680edd92-20a9-4aff-adb5-3499f7db68de None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:05.641916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34545ac1-0fc9-4e75-b40a-912979241c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:05.642602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34545ac1-0fc9-4e75-b40a-912979241c1a admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:05.643043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34545ac1-0fc9-4e75-b40a-912979241c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:05.643240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34545ac1-0fc9-4e75-b40a-912979241c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:05.643948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34545ac1-0fc9-4e75-b40a-912979241c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:05.646003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34545ac1-0fc9-4e75-b40a-912979241c1a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:05.784521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 980/1943] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:05 2022] POST /identity/v3/projects => generated 409 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:05.791050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:05.799585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070392e-d8cf-4da0-bf6a-92d6e73ef281 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:05.861003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ca68e-d829-426d-817a-042538c252a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:05.861692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ca68e-d829-426d-817a-042538c252a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:05.862009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ca68e-d829-426d-817a-042538c252a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:05.862289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ca68e-d829-426d-817a-042538c252a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:05.862998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ca68e-d829-426d-817a-042538c252a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:05.864938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ca68e-d829-426d-817a-042538c252a8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:05.973725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 964/1944] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:05 2022] POST /identity/v3/users => generated 305 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:05.977767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34545ac1-0fc9-4e75-b40a-912979241c1a admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:05.995364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34545ac1-0fc9-4e75-b40a-912979241c1a admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:06.002341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf328c0-ddbf-4497-a521-ae5b3f004241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:06.003153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf328c0-ddbf-4497-a521-ae5b3f004241 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:06.003517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf328c0-ddbf-4497-a521-ae5b3f004241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:06.003869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf328c0-ddbf-4497-a521-ae5b3f004241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:06.004685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf328c0-ddbf-4497-a521-ae5b3f004241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:06.007269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf328c0-ddbf-4497-a521-ae5b3f004241 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:06.022607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 981/1945] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:06.030206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ca68e-d829-426d-817a-042538c252a8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:06.040995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ca68e-d829-426d-817a-042538c252a8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:06.049307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a91aa2-d0c5-4467-a7cf-51f22c2125de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:06.050084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a91aa2-d0c5-4467-a7cf-51f22c2125de admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:06.050371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a91aa2-d0c5-4467-a7cf-51f22c2125de admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:06.050653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a91aa2-d0c5-4467-a7cf-51f22c2125de admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:06.051545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a91aa2-d0c5-4467-a7cf-51f22c2125de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:06.053174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a91aa2-d0c5-4467-a7cf-51f22c2125de admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:06.065369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 965/1946] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:06.069933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf328c0-ddbf-4497-a521-ae5b3f004241 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:06.086702 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf328c0-ddbf-4497-a521-ae5b3f004241 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:06.098972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cc634-ca77-4078-a03c-1f3fb1bc1840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:06.100198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cc634-ca77-4078-a03c-1f3fb1bc1840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:06.116178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cc634-ca77-4078-a03c-1f3fb1bc1840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:06.116479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cc634-ca77-4078-a03c-1f3fb1bc1840 admin admin] PATH_INFO: `/v3/projects/401a52cf9191439fa8a693ceb9c302ab/users/ffd04d65c5204adb800a5c01f3451957/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:06.141587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cc634-ca77-4078-a03c-1f3fb1bc1840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401a52cf9191439fa8a693ceb9c302ab, user_id=ffd04d65c5204adb800a5c01f3451957, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:06.144669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cc634-ca77-4078-a03c-1f3fb1bc1840 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:06.262636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 982/1947] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:06 2022] PUT /identity/v3/projects/401a52cf9191439fa8a693ceb9c302ab/users/ffd04d65c5204adb800a5c01f3451957/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:06.273992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285bf87-6390-4dec-82ce-9151c0618555 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:06.274677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285bf87-6390-4dec-82ce-9151c0618555 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:06.274677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285bf87-6390-4dec-82ce-9151c0618555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:06.309236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9285bf87-6390-4dec-82ce-9151c0618555 None None] MFA Rules not processed for user `ffd04d65c5204adb800a5c01f3451957`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:06.310479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9285bf87-6390-4dec-82ce-9151c0618555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:06.375675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 966/1948] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:06.380227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cc634-ca77-4078-a03c-1f3fb1bc1840 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:06.460305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cc634-ca77-4078-a03c-1f3fb1bc1840 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:06.468125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8d435-c364-46a3-ab8a-7fb3071e7961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:06.468812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8d435-c364-46a3-ab8a-7fb3071e7961 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:06.478607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8d435-c364-46a3-ab8a-7fb3071e7961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:06.478607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8d435-c364-46a3-ab8a-7fb3071e7961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:06.478607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8d435-c364-46a3-ab8a-7fb3071e7961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:06.478607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8d435-c364-46a3-ab8a-7fb3071e7961 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:06.620652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 983/1949] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:06 2022] POST /identity/v3/projects => generated 407 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:06.625550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285bf87-6390-4dec-82ce-9151c0618555 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:06.642351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285bf87-6390-4dec-82ce-9151c0618555 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:06.703737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456f4f1-d88a-4567-b68b-1403386759bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:06.704442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f4f1-d88a-4567-b68b-1403386759bb admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:06.704731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f4f1-d88a-4567-b68b-1403386759bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:06.705009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456f4f1-d88a-4567-b68b-1403386759bb admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:06.705715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456f4f1-d88a-4567-b68b-1403386759bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:06.707678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456f4f1-d88a-4567-b68b-1403386759bb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:06.838566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 967/1950] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:06 2022] POST /identity/v3/users => generated 310 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:06.846139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8d435-c364-46a3-ab8a-7fb3071e7961 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:06.859393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8d435-c364-46a3-ab8a-7fb3071e7961 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:06.865237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d7f8d-9392-48f2-890c-cfbe6ad0871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:06.866046 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d7f8d-9392-48f2-890c-cfbe6ad0871f admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:06.866366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d7f8d-9392-48f2-890c-cfbe6ad0871f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:06.866653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d7f8d-9392-48f2-890c-cfbe6ad0871f admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:06.867387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d7f8d-9392-48f2-890c-cfbe6ad0871f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:06.869411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d7f8d-9392-48f2-890c-cfbe6ad0871f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:06.888627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 984/1951] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:06.892420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456f4f1-d88a-4567-b68b-1403386759bb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:06.906765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456f4f1-d88a-4567-b68b-1403386759bb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:06.911884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e6473-0573-43d5-8cbc-b2a44827d460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:06.912788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6473-0573-43d5-8cbc-b2a44827d460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:06.913072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6473-0573-43d5-8cbc-b2a44827d460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:06.913353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6473-0573-43d5-8cbc-b2a44827d460 admin admin] PATH_INFO: `/v3/projects/60b45795c6664a68aaa9a53e3141841b/users/790719d467cf458588c3400e6d50e431/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:06.932632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e6473-0573-43d5-8cbc-b2a44827d460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b45795c6664a68aaa9a53e3141841b, user_id=790719d467cf458588c3400e6d50e431, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:06.936067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e6473-0573-43d5-8cbc-b2a44827d460 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:07.037906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 968/1952] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:06 2022] PUT /identity/v3/projects/60b45795c6664a68aaa9a53e3141841b/users/790719d467cf458588c3400e6d50e431/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:07.041774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d7f8d-9392-48f2-890c-cfbe6ad0871f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:07.059028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d7f8d-9392-48f2-890c-cfbe6ad0871f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:07.125674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:07.126446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:07.126740 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:07.127070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:07.127870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:07.129950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:07.144494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 985/1953] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:07.145688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e6473-0573-43d5-8cbc-b2a44827d460 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:07.161239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e6473-0573-43d5-8cbc-b2a44827d460 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:07.173649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57462d-1016-465a-8a88-0fd608005094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:07.173649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57462d-1016-465a-8a88-0fd608005094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:07.173649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57462d-1016-465a-8a88-0fd608005094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:07.178033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57462d-1016-465a-8a88-0fd608005094 admin admin] PATH_INFO: `/v3/projects/60b45795c6664a68aaa9a53e3141841b/users/790719d467cf458588c3400e6d50e431/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:07.192661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57462d-1016-465a-8a88-0fd608005094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b45795c6664a68aaa9a53e3141841b, user_id=790719d467cf458588c3400e6d50e431, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:07.195329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57462d-1016-465a-8a88-0fd608005094 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:07.204017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:07.206457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:07.220660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 969/1954] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:07 2022] PUT /identity/v3/projects/60b45795c6664a68aaa9a53e3141841b/users/790719d467cf458588c3400e6d50e431/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:07.232582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751184b-4f1a-4892-99e0-07527f171b3b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:07.232864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751184b-4f1a-4892-99e0-07527f171b3b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:07.233269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751184b-4f1a-4892-99e0-07527f171b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:07.270048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8751184b-4f1a-4892-99e0-07527f171b3b None None] MFA Rules not processed for user `790719d467cf458588c3400e6d50e431`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:07.271822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8751184b-4f1a-4892-99e0-07527f171b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:07.332813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f4d26-4fc7-4525-a97c-ace4c2d9db64 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:07.383556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 970/1955] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:07 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:07.421502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:07.422631 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:07.423131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:07.423697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:07.425907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:07.529976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:07.537819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:07.552205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751184b-4f1a-4892-99e0-07527f171b3b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:07.554372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8751184b-4f1a-4892-99e0-07527f171b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:07.557437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 986/1956] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:07.592674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751184b-4f1a-4892-99e0-07527f171b3b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:07.600899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:07.602535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:07.602827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:07.603111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:07.629643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:07.635425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:07.663416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 971/1957] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:07 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:08.322567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:08.332129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca8e7d-4d62-4cef-b088-0dd7d860a8f8 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:08.336105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c69ffb-d73c-48f9-9c93-4abac1f4be32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:08.336749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c69ffb-d73c-48f9-9c93-4abac1f4be32 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:08.340450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c69ffb-d73c-48f9-9c93-4abac1f4be32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:08.340450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c69ffb-d73c-48f9-9c93-4abac1f4be32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:08.392510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c69ffb-d73c-48f9-9c93-4abac1f4be32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:08.393893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c69ffb-d73c-48f9-9c93-4abac1f4be32 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:08.403614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 987/1958] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:10.658957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:10.660470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:10.734695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd8c8d-a07b-4664-a0e3-c4bf3f8b2373 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:10.739772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:10.741065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:10.741647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:10.742095 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] PATH_INFO: `/v3/users/4601bfe2b69b4b2fb268a593bebbe2a7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:10.743426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4601bfe2b69b4b2fb268a593bebbe2a7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:10.745381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:10.945046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4601bfe2b69b4b2fb268a593bebbe2a7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:10.945883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4601bfe2b69b4b2fb268a593bebbe2a7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:10.946747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4601bfe2b69b4b2fb268a593bebbe2a7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:10.955444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4601bfe2b69b4b2fb268a593bebbe2a7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:11.028297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4601bfe2b69b4b2fb268a593bebbe2a7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:11.047129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 972/1959] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:10 2022] DELETE /identity/v3/users/4601bfe2b69b4b2fb268a593bebbe2a7 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:11.051489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98734d7-3174-4934-828b-c55136672b2d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:11.051969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98734d7-3174-4934-828b-c55136672b2d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:11.052183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98734d7-3174-4934-828b-c55136672b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:11.068823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f98734d7-3174-4934-828b-c55136672b2d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:11.070190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f98734d7-3174-4934-828b-c55136672b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:11.128013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 988/1960] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:11.137573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:11.138934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:11.195463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fa346-5f16-48a4-85b2-a1ed8a0eb8cd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:11.199384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64450b7a-3136-436d-9a07-7a91601f9984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:11.200035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64450b7a-3136-436d-9a07-7a91601f9984 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:11.200323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64450b7a-3136-436d-9a07-7a91601f9984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:11.200600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64450b7a-3136-436d-9a07-7a91601f9984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:11.210106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64450b7a-3136-436d-9a07-7a91601f9984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:11.211549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64450b7a-3136-436d-9a07-7a91601f9984 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:11.222493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 973/1961] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:11.991325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98734d7-3174-4934-828b-c55136672b2d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:11.992743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f98734d7-3174-4934-828b-c55136672b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:12.005937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98734d7-3174-4934-828b-c55136672b2d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:12.010845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:12.012359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:12.012644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:12.012920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] PATH_INFO: `/v3/projects/248d93ad559f416893e1a8d97ab0a45f` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:12.014173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] RBAC: Authorizing `identity:delete_project(project_id=248d93ad559f416893e1a8d97ab0a45f)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:12.016096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:12.166072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '248d93ad559f416893e1a8d97ab0a45f', 'request_id': 'req-d6309a13-c1ef-4a3f-8715-7dca78c3759d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.172583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '248d93ad559f416893e1a8d97ab0a45f', 'request_id': 'req-d6309a13-c1ef-4a3f-8715-7dca78c3759d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.254174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '248d93ad559f416893e1a8d97ab0a45f', 'request_id': 'req-d6309a13-c1ef-4a3f-8715-7dca78c3759d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.266176 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] The token cache is being invalidate because project 248d93ad559f416893e1a8d97ab0a45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:12.266697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.267340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.269038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 989/1962] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:11 2022] DELETE /identity/v3/projects/248d93ad559f416893e1a8d97ab0a45f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:12.272932 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64450b7a-3136-436d-9a07-7a91601f9984 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:12.282673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64450b7a-3136-436d-9a07-7a91601f9984 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:12.343206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:12.344221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:12.344590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:12.344878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] PATH_INFO: `/v3/users/9a1094ec56f7413989f87af30f1f2e63` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:12.346300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a1094ec56f7413989f87af30f1f2e63)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:12.348241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:12.548879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a1094ec56f7413989f87af30f1f2e63'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.551023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a1094ec56f7413989f87af30f1f2e63'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.551023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a1094ec56f7413989f87af30f1f2e63'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.556976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a1094ec56f7413989f87af30f1f2e63'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.632840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a1094ec56f7413989f87af30f1f2e63'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:12.649903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 974/1963] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:12 2022] DELETE /identity/v3/users/9a1094ec56f7413989f87af30f1f2e63 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:13.095497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:13.096909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:13.111373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6309a13-c1ef-4a3f-8715-7dca78c3759d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:13.166005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:13.166942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:13.167510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:13.167510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] PATH_INFO: `/v3/projects/48bf5d49bdd440bca669489e9d26d9c0` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:13.169295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=48bf5d49bdd440bca669489e9d26d9c0)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:13.171334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:13.273214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48bf5d49bdd440bca669489e9d26d9c0', 'request_id': 'req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.279294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48bf5d49bdd440bca669489e9d26d9c0', 'request_id': 'req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.319172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48bf5d49bdd440bca669489e9d26d9c0', 'request_id': 'req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.331107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] The token cache is being invalidate because project 48bf5d49bdd440bca669489e9d26d9c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:13.331652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.332292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.333853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 990/1964] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:13 2022] DELETE /identity/v3/projects/48bf5d49bdd440bca669489e9d26d9c0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:13.337880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:13.339383 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:13.352282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f41119-81d8-4bff-9e65-48fb01101ff6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:13.416661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:13.417874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:13.418760 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:13.419133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] PATH_INFO: `/v3/users/81fe76d3a3944648ad937eb9d41b59f7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:13.420435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81fe76d3a3944648ad937eb9d41b59f7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:13.422673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:13.611330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81fe76d3a3944648ad937eb9d41b59f7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.611985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81fe76d3a3944648ad937eb9d41b59f7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.612585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81fe76d3a3944648ad937eb9d41b59f7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.619508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81fe76d3a3944648ad937eb9d41b59f7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.689939 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5c307cf2-e4b4-4b5e-b0a6-7df97628eb13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81fe76d3a3944648ad937eb9d41b59f7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:13.707568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 975/1965] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:13 2022] DELETE /identity/v3/users/81fe76d3a3944648ad937eb9d41b59f7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:14.152675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:14.159121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:14.167945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065700c-4dab-4fd5-9a6f-c3e73e8d2a6f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:14.224364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:14.225291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:14.225570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:14.238455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] PATH_INFO: `/v3/projects/d65bf918451540e9890385fa05d1b066` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:14.238455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65bf918451540e9890385fa05d1b066)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:14.238455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:14.376192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65bf918451540e9890385fa05d1b066', 'request_id': 'req-0d11ffe9-1c65-49cd-aa36-7e139069e70d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:14.381270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65bf918451540e9890385fa05d1b066', 'request_id': 'req-0d11ffe9-1c65-49cd-aa36-7e139069e70d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:14.467567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65bf918451540e9890385fa05d1b066', 'request_id': 'req-0d11ffe9-1c65-49cd-aa36-7e139069e70d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:14.479374 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] The token cache is being invalidate because project d65bf918451540e9890385fa05d1b066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:14.480058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:14.481247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0d11ffe9-1c65-49cd-aa36-7e139069e70d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:14.482668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 991/1966] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:14 2022] DELETE /identity/v3/projects/d65bf918451540e9890385fa05d1b066 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:14.488634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae641a-8dea-4f24-8a3b-20c4f89e0bcd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:14.489546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae641a-8dea-4f24-8a3b-20c4f89e0bcd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:14.489864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae641a-8dea-4f24-8a3b-20c4f89e0bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:14.509983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0fae641a-8dea-4f24-8a3b-20c4f89e0bcd None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:14.511194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0fae641a-8dea-4f24-8a3b-20c4f89e0bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:14.581338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 976/1967] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:14.587952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d149-ea77-40f3-9f24-432c8d4dd2fe None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:14.588246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d149-ea77-40f3-9f24-432c8d4dd2fe None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:14.588524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d149-ea77-40f3-9f24-432c8d4dd2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:14.607776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0b26d149-ea77-40f3-9f24-432c8d4dd2fe None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:14.608982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0b26d149-ea77-40f3-9f24-432c8d4dd2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:14.617318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 992/1968] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:14.621710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae641a-8dea-4f24-8a3b-20c4f89e0bcd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:14.631636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae641a-8dea-4f24-8a3b-20c4f89e0bcd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:14.637396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:14.641039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:14.641039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:14.641039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:14.641039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:14.641039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:14.641896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:14.659135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 977/1969] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:26:14.660462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26d149-ea77-40f3-9f24-432c8d4dd2fe None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:14.669743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26d149-ea77-40f3-9f24-432c8d4dd2fe None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:14.674598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56169f-e7f9-4e24-af21-c4b14752fab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:14.675257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56169f-e7f9-4e24-af21-c4b14752fab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:14.675547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56169f-e7f9-4e24-af21-c4b14752fab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:14.675825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56169f-e7f9-4e24-af21-c4b14752fab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:14.676516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56169f-e7f9-4e24-af21-c4b14752fab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:14.678501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56169f-e7f9-4e24-af21-c4b14752fab9 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:14.766776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 993/1970] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:14 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:14.770923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:14.781617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b96de4-f005-4b2f-9b40-d1af531980d9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:14.838118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332c11d-e3bf-464b-b998-215b92363123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:14.838786 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332c11d-e3bf-464b-b998-215b92363123 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:14.839068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332c11d-e3bf-464b-b998-215b92363123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:14.839339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332c11d-e3bf-464b-b998-215b92363123 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:14.840017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332c11d-e3bf-464b-b998-215b92363123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:14.841983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332c11d-e3bf-464b-b998-215b92363123 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:14.952731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 978/1971] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:14 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:14.956701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56169f-e7f9-4e24-af21-c4b14752fab9 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:14.966888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56169f-e7f9-4e24-af21-c4b14752fab9 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:14.971498 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6cc6a-0426-40a4-9ac7-ba446fa2eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:14.972161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cc6a-0426-40a4-9ac7-ba446fa2eabe admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:14.972446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cc6a-0426-40a4-9ac7-ba446fa2eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:14.972725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cc6a-0426-40a4-9ac7-ba446fa2eabe admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:14.973455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6cc6a-0426-40a4-9ac7-ba446fa2eabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:14.975131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6cc6a-0426-40a4-9ac7-ba446fa2eabe admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:14.985599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 994/1972] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:14.989353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332c11d-e3bf-464b-b998-215b92363123 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:14.999805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332c11d-e3bf-464b-b998-215b92363123 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:15.004395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca7929-0582-490e-9476-afc4c67f0ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:15.005054 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca7929-0582-490e-9476-afc4c67f0ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.005341 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca7929-0582-490e-9476-afc4c67f0ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.005624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca7929-0582-490e-9476-afc4c67f0ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.006384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca7929-0582-490e-9476-afc4c67f0ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:15.007983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca7929-0582-490e-9476-afc4c67f0ae9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:15.018254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 979/1973] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:15.022013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6cc6a-0426-40a4-9ac7-ba446fa2eabe admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:15.032299 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6cc6a-0426-40a4-9ac7-ba446fa2eabe admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:15.037132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b76563-f5fc-4a7b-9ae1-e786e910581d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:15.038044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b76563-f5fc-4a7b-9ae1-e786e910581d admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.038329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b76563-f5fc-4a7b-9ae1-e786e910581d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.038606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b76563-f5fc-4a7b-9ae1-e786e910581d admin admin] PATH_INFO: `/v3/projects/e4283617dcb24d1b90259ac807ed087b/users/6335c39310f443fe842e58de0d616783/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.054941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b76563-f5fc-4a7b-9ae1-e786e910581d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4283617dcb24d1b90259ac807ed087b, user_id=6335c39310f443fe842e58de0d616783, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:15.057656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b76563-f5fc-4a7b-9ae1-e786e910581d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:15.186409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 995/1974] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:15 2022] PUT /identity/v3/projects/e4283617dcb24d1b90259ac807ed087b/users/6335c39310f443fe842e58de0d616783/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:15.197717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdc5f-948f-4968-ac66-328d760ced91 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.204163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdc5f-948f-4968-ac66-328d760ced91 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.204163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdc5f-948f-4968-ac66-328d760ced91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.223915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bd8fdc5f-948f-4968-ac66-328d760ced91 None None] MFA Rules not processed for user `6335c39310f443fe842e58de0d616783`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:15.225139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd8fdc5f-948f-4968-ac66-328d760ced91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:15.291344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 980/1975] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:15.293404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b76563-f5fc-4a7b-9ae1-e786e910581d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:15.357151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b76563-f5fc-4a7b-9ae1-e786e910581d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:15.361983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45b6c6-e7ff-46c3-9b63-6807fdcdfd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:15.362646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b6c6-e7ff-46c3-9b63-6807fdcdfd32 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.362933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b6c6-e7ff-46c3-9b63-6807fdcdfd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.363213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b6c6-e7ff-46c3-9b63-6807fdcdfd32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.363943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45b6c6-e7ff-46c3-9b63-6807fdcdfd32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:15.365982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45b6c6-e7ff-46c3-9b63-6807fdcdfd32 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:15.467399 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 996/1976] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:15 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:15.471323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fdc5f-948f-4968-ac66-328d760ced91 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:15.482770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fdc5f-948f-4968-ac66-328d760ced91 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:15.537421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c75e-ed36-43be-8bde-b23850d9886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:15.538123 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c75e-ed36-43be-8bde-b23850d9886f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.538404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c75e-ed36-43be-8bde-b23850d9886f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.538685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01c75e-ed36-43be-8bde-b23850d9886f admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.539377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01c75e-ed36-43be-8bde-b23850d9886f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:15.541329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01c75e-ed36-43be-8bde-b23850d9886f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:15.674813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 981/1977] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:15 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:15.678527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45b6c6-e7ff-46c3-9b63-6807fdcdfd32 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:15.689258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45b6c6-e7ff-46c3-9b63-6807fdcdfd32 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:15.694088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77c8c-f9d0-4ebb-9c10-67a3597c3cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:15.694940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77c8c-f9d0-4ebb-9c10-67a3597c3cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.695230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77c8c-f9d0-4ebb-9c10-67a3597c3cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.695507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77c8c-f9d0-4ebb-9c10-67a3597c3cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.696492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba77c8c-f9d0-4ebb-9c10-67a3597c3cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:15.698128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba77c8c-f9d0-4ebb-9c10-67a3597c3cdc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:15.708860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 997/1978] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:15.712906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c75e-ed36-43be-8bde-b23850d9886f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:15.724209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01c75e-ed36-43be-8bde-b23850d9886f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:15.730477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176ae7-3a37-4915-a119-255e72ead161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:15.731872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176ae7-3a37-4915-a119-255e72ead161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.732158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176ae7-3a37-4915-a119-255e72ead161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.732472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176ae7-3a37-4915-a119-255e72ead161 admin admin] PATH_INFO: `/v3/projects/e31a12ca8f6c43149a31520f8fd4065e/users/a387b4430dd74785b90e87432c4bcfdf/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.749314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61176ae7-3a37-4915-a119-255e72ead161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31a12ca8f6c43149a31520f8fd4065e, user_id=a387b4430dd74785b90e87432c4bcfdf, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:15.752194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61176ae7-3a37-4915-a119-255e72ead161 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:15.863863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 982/1979] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:15 2022] PUT /identity/v3/projects/e31a12ca8f6c43149a31520f8fd4065e/users/a387b4430dd74785b90e87432c4bcfdf/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:15.867890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77c8c-f9d0-4ebb-9c10-67a3597c3cdc admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:15.878738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77c8c-f9d0-4ebb-9c10-67a3597c3cdc admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:15.940504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0de12-8514-4bd7-a6aa-64f7de60be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:15.941180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0de12-8514-4bd7-a6aa-64f7de60be96 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.941466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0de12-8514-4bd7-a6aa-64f7de60be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.941745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0de12-8514-4bd7-a6aa-64f7de60be96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.942502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0de12-8514-4bd7-a6aa-64f7de60be96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:15.944153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0de12-8514-4bd7-a6aa-64f7de60be96 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:15.954194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 998/1980] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:15.958457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176ae7-3a37-4915-a119-255e72ead161 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:15.968585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176ae7-3a37-4915-a119-255e72ead161 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:15.974422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:15.974422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:15.974422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:15.974734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] PATH_INFO: `/v3/projects/e31a12ca8f6c43149a31520f8fd4065e/users/a387b4430dd74785b90e87432c4bcfdf/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:15.976735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31a12ca8f6c43149a31520f8fd4065e, user_id=a387b4430dd74785b90e87432c4bcfdf, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:15.979447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:15.990009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 983/1981] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:15 2022] PUT /identity/v3/projects/e31a12ca8f6c43149a31520f8fd4065e/users/a387b4430dd74785b90e87432c4bcfdf/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:16.001456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2348935-9034-4c49-8307-c3c58bfc4883 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:16.001848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2348935-9034-4c49-8307-c3c58bfc4883 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:16.002270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2348935-9034-4c49-8307-c3c58bfc4883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:16.030768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d2348935-9034-4c49-8307-c3c58bfc4883 None None] MFA Rules not processed for user `a387b4430dd74785b90e87432c4bcfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:16.032068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d2348935-9034-4c49-8307-c3c58bfc4883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:16.099056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 999/1982] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:16.110989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:16.112673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:16.168763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31184ddd-080c-4bed-a7ad-8309f680c841 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:16.172425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:16.173099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:16.173385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:16.173662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:16.183002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:16.184995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:16.194443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 984/1983] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:16.447233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2348935-9034-4c49-8307-c3c58bfc4883 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:16.456801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2348935-9034-4c49-8307-c3c58bfc4883 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:16.460332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65bd92-94ef-47bc-b480-c913437b8e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:16.460980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65bd92-94ef-47bc-b480-c913437b8e50 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:16.461291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65bd92-94ef-47bc-b480-c913437b8e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:16.461565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65bd92-94ef-47bc-b480-c913437b8e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:16.523930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65bd92-94ef-47bc-b480-c913437b8e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:16.525387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65bd92-94ef-47bc-b480-c913437b8e50 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:16.535141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1000/1984] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:21.428539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:21.431235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:21.496654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653df83e-79e7-4eb4-b883-88612c0a8fd0 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:21.501285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:21.502190 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:21.502475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:21.502756 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] PATH_INFO: `/v3/users/ffd04d65c5204adb800a5c01f3451957` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:21.503948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd04d65c5204adb800a5c01f3451957)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:21.505990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:22.031865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd04d65c5204adb800a5c01f3451957'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:22.032535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd04d65c5204adb800a5c01f3451957'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:22.033144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd04d65c5204adb800a5c01f3451957'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:22.048092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd04d65c5204adb800a5c01f3451957'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:22.158931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd04d65c5204adb800a5c01f3451957'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:22.176431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 985/1985] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:21 2022] DELETE /identity/v3/users/ffd04d65c5204adb800a5c01f3451957 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:22.181427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e86b6-d427-4cb6-af13-6e301435a6ff None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:22.181929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e86b6-d427-4cb6-af13-6e301435a6ff None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:22.182017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e86b6-d427-4cb6-af13-6e301435a6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:22.201925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ee9e86b6-d427-4cb6-af13-6e301435a6ff None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:22.207881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ee9e86b6-d427-4cb6-af13-6e301435a6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:22.277986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1001/1986] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:22.287592 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:22.289121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:22.347370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde09645-3c44-4737-b21a-04af472269ba admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:22.351055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4da3d-11bb-41b2-a54d-835e4ba1dd5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:22.351708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4da3d-11bb-41b2-a54d-835e4ba1dd5c service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:22.352035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4da3d-11bb-41b2-a54d-835e4ba1dd5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:22.352315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4da3d-11bb-41b2-a54d-835e4ba1dd5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:22.362308 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4da3d-11bb-41b2-a54d-835e4ba1dd5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:22.363494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4da3d-11bb-41b2-a54d-835e4ba1dd5c service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:22.375167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 986/1987] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:22.905304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e86b6-d427-4cb6-af13-6e301435a6ff None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:22.912317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ee9e86b6-d427-4cb6-af13-6e301435a6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:22.938440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e86b6-d427-4cb6-af13-6e301435a6ff None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:22.944343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:22.945457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:22.945741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:22.946067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] PATH_INFO: `/v3/projects/401a52cf9191439fa8a693ceb9c302ab` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:22.947522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] RBAC: Authorizing `identity:delete_project(project_id=401a52cf9191439fa8a693ceb9c302ab)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:22.949482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:23.082156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401a52cf9191439fa8a693ceb9c302ab', 'request_id': 'req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.087444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401a52cf9191439fa8a693ceb9c302ab', 'request_id': 'req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.193635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401a52cf9191439fa8a693ceb9c302ab', 'request_id': 'req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.205266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] The token cache is being invalidate because project 401a52cf9191439fa8a693ceb9c302ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:23.205554 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.206207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.207765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1002/1988] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:22 2022] DELETE /identity/v3/projects/401a52cf9191439fa8a693ceb9c302ab => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:23.211768 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4da3d-11bb-41b2-a54d-835e4ba1dd5c service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:23.221690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4da3d-11bb-41b2-a54d-835e4ba1dd5c service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:23.293780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:23.293780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:23.293780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:23.293780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] PATH_INFO: `/v3/users/790719d467cf458588c3400e6d50e431` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:23.294087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=790719d467cf458588c3400e6d50e431)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:23.294087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:23.464258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790719d467cf458588c3400e6d50e431'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.465421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790719d467cf458588c3400e6d50e431'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.466062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790719d467cf458588c3400e6d50e431'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.473129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790719d467cf458588c3400e6d50e431'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.576084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790719d467cf458588c3400e6d50e431'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:23.593743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 987/1989] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:23 2022] DELETE /identity/v3/users/790719d467cf458588c3400e6d50e431 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:23.953940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:23.955495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:23.971514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2809a-68c2-40a1-b0ee-3f85e7ee691d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:24.036453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:24.037528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.037911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.038257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] PATH_INFO: `/v3/users/e0cd13c6d5a540bbb8dd02bcfedeb211` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.039536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0cd13c6d5a540bbb8dd02bcfedeb211)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:24.041561 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:24.052718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:24.054201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:24.068219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd495f-1349-4a08-a6b4-37d9bbc1e8cc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:24.080998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:24.081926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.082376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.082831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] PATH_INFO: `/v3/projects/60b45795c6664a68aaa9a53e3141841b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.084231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b45795c6664a68aaa9a53e3141841b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:24.088354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:24.261167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0cd13c6d5a540bbb8dd02bcfedeb211'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.261737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b45795c6664a68aaa9a53e3141841b', 'request_id': 'req-41551a18-196d-45b3-95fd-9de934028145'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.262167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0cd13c6d5a540bbb8dd02bcfedeb211'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.266716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b45795c6664a68aaa9a53e3141841b', 'request_id': 'req-41551a18-196d-45b3-95fd-9de934028145'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.330471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0cd13c6d5a540bbb8dd02bcfedeb211'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.337731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0cd13c6d5a540bbb8dd02bcfedeb211'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.338395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-b6c522b5-25b8-4fbb-8660-59423e6cbf02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0cd13c6d5a540bbb8dd02bcfedeb211'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.361336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1003/1990] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:23 2022] DELETE /identity/v3/users/e0cd13c6d5a540bbb8dd02bcfedeb211 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:24.365559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1034a63-b5fe-4e74-a256-f81ce7b605df None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.370379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1034a63-b5fe-4e74-a256-f81ce7b605df None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.370379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1034a63-b5fe-4e74-a256-f81ce7b605df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.383560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f1034a63-b5fe-4e74-a256-f81ce7b605df None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:24.386711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f1034a63-b5fe-4e74-a256-f81ce7b605df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:24.402551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b45795c6664a68aaa9a53e3141841b', 'request_id': 'req-41551a18-196d-45b3-95fd-9de934028145'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.414859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] The token cache is being invalidate because project 60b45795c6664a68aaa9a53e3141841b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:24.415367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.416200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-41551a18-196d-45b3-95fd-9de934028145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:24.418500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 988/1991] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:24 2022] DELETE /identity/v3/projects/60b45795c6664a68aaa9a53e3141841b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:24.427062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ef641-7775-4913-b528-ca6e6e02ffd7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.427782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ef641-7775-4913-b528-ca6e6e02ffd7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.428042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ef641-7775-4913-b528-ca6e6e02ffd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.450691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fe1ef641-7775-4913-b528-ca6e6e02ffd7 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:24.452049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fe1ef641-7775-4913-b528-ca6e6e02ffd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:24.461561 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1004/1992] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:24.472389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1034a63-b5fe-4e74-a256-f81ce7b605df None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:24.475389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f1034a63-b5fe-4e74-a256-f81ce7b605df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:24.526024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 989/1993] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:24.531218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893814b8-63cc-4dc3-9f68-deb653b0368e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.531508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893814b8-63cc-4dc3-9f68-deb653b0368e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.531843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893814b8-63cc-4dc3-9f68-deb653b0368e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.554433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1034a63-b5fe-4e74-a256-f81ce7b605df None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:24.555670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-893814b8-63cc-4dc3-9f68-deb653b0368e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:24.556938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-893814b8-63cc-4dc3-9f68-deb653b0368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:24.558957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9dc65-d206-439c-85d1-7ac5290f5307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:24.559610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9dc65-d206-439c-85d1-7ac5290f5307 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.559893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9dc65-d206-439c-85d1-7ac5290f5307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.560170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9dc65-d206-439c-85d1-7ac5290f5307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.565221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 990/1994] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:24.569626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893814b8-63cc-4dc3-9f68-deb653b0368e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:24.569820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9dc65-d206-439c-85d1-7ac5290f5307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:24.571119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9dc65-d206-439c-85d1-7ac5290f5307 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:24.579175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893814b8-63cc-4dc3-9f68-deb653b0368e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:24.581326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1005/1995] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:24.584082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:24.584737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.585021 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.585299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.586318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:24.586632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:24.588246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:24.604985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 991/1996] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:24.606724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9dc65-d206-439c-85d1-7ac5290f5307 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:24.616143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9dc65-d206-439c-85d1-7ac5290f5307 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:24.620744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d212f9e-b2d3-4007-af5a-1a4857c18b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:24.621416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212f9e-b2d3-4007-af5a-1a4857c18b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.621694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212f9e-b2d3-4007-af5a-1a4857c18b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.621998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d212f9e-b2d3-4007-af5a-1a4857c18b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.622712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d212f9e-b2d3-4007-af5a-1a4857c18b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:24.624609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d212f9e-b2d3-4007-af5a-1a4857c18b09 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:24.743795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1006/1997] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:24 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:24.747989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:24.757862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba5380-0606-4bb1-9676-52aefd211dec admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:24.817564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2d885-f44f-470b-907f-cd60fd1c425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:24.818280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2d885-f44f-470b-907f-cd60fd1c425c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.818772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2d885-f44f-470b-907f-cd60fd1c425c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.819055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2d885-f44f-470b-907f-cd60fd1c425c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.819753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2d885-f44f-470b-907f-cd60fd1c425c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:24.821998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2d885-f44f-470b-907f-cd60fd1c425c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:24.937668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 992/1998] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:24 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:24.940402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d212f9e-b2d3-4007-af5a-1a4857c18b09 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:24.956321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d212f9e-b2d3-4007-af5a-1a4857c18b09 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:24.969260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295cb35-9c93-407d-98ec-35c2e516e4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:24.970004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295cb35-9c93-407d-98ec-35c2e516e4ee admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:24.970297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295cb35-9c93-407d-98ec-35c2e516e4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:24.970576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295cb35-9c93-407d-98ec-35c2e516e4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:24.971314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295cb35-9c93-407d-98ec-35c2e516e4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:24.972929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295cb35-9c93-407d-98ec-35c2e516e4ee admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:24.995773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1007/1999] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:24.999830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2d885-f44f-470b-907f-cd60fd1c425c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:25.014067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2d885-f44f-470b-907f-cd60fd1c425c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:25.019771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:25.020503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.020788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.021069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.021829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:25.023594 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:25.036208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 993/2000] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:25.041188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295cb35-9c93-407d-98ec-35c2e516e4ee admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:25.057196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295cb35-9c93-407d-98ec-35c2e516e4ee admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:25.064973 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:25.066443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.066972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.067470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] PATH_INFO: `/v3/projects/564789086e0f4c91a2159872edba8e91/users/d2f18afce16e43df97efa0f18c47fb2b/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.088351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=564789086e0f4c91a2159872edba8e91, user_id=d2f18afce16e43df97efa0f18c47fb2b, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:25.093834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:25.099711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:25.102374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:25.121610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7d9d6-b421-47bb-9a0b-e81007bd73c8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:25.126060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:25.127094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.127683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.127981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] PATH_INFO: `/v3/users/80623f3527844489a5e31a786080fa10` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.129324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] RBAC: Authorizing `identity:delete_user(user_id=80623f3527844489a5e31a786080fa10)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:25.131307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:25.212215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1008/2001] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:25 2022] PUT /identity/v3/projects/564789086e0f4c91a2159872edba8e91/users/d2f18afce16e43df97efa0f18c47fb2b/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:25.218558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:25.222194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:25.303487 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5540a50-d1e2-4c56-9ff7-9965e3e32acc admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:25.308956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:25.309887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.310170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.310442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] PATH_INFO: `/v3/users/6335c39310f443fe842e58de0d616783` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.311674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6335c39310f443fe842e58de0d616783)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:25.313598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:25.362237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80623f3527844489a5e31a786080fa10'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.362894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80623f3527844489a5e31a786080fa10'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.363507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80623f3527844489a5e31a786080fa10'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.374263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80623f3527844489a5e31a786080fa10'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.519258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80623f3527844489a5e31a786080fa10'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.538288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 994/2002] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:25 2022] DELETE /identity/v3/users/80623f3527844489a5e31a786080fa10 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:25.539134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:25.540850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:25.608278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd06aa-7cb9-4336-84ea-94a0423d105d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:25.616946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:25.621971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.621971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.621971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] PATH_INFO: `/v3/projects/4c9872b9ac4647c4bb98bc8ec82a6ca5` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.621971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9872b9ac4647c4bb98bc8ec82a6ca5)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:25.623069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:25.628387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6335c39310f443fe842e58de0d616783'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.629058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6335c39310f443fe842e58de0d616783'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.696261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6335c39310f443fe842e58de0d616783'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.705388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6335c39310f443fe842e58de0d616783'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.706140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d6e68c1-0bd5-45ba-951c-fbde9e4a1871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6335c39310f443fe842e58de0d616783'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.724168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1009/2003] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:25 2022] DELETE /identity/v3/users/6335c39310f443fe842e58de0d616783 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:25.725847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8513f4e-d293-4388-b869-7c8f66032efc None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.726201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8513f4e-d293-4388-b869-7c8f66032efc None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.726486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8513f4e-d293-4388-b869-7c8f66032efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.749582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9872b9ac4647c4bb98bc8ec82a6ca5', 'request_id': 'req-540b6296-98ca-4d06-b20a-c01e6b88d129'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.755644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9872b9ac4647c4bb98bc8ec82a6ca5', 'request_id': 'req-540b6296-98ca-4d06-b20a-c01e6b88d129'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.765927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b8513f4e-d293-4388-b869-7c8f66032efc None None] MFA Rules not processed for user `d2f18afce16e43df97efa0f18c47fb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:25.767158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b8513f4e-d293-4388-b869-7c8f66032efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:25.844850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1010/2004] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:25.846289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d987c-2e15-47cf-8854-3a52242640f6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.846583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d987c-2e15-47cf-8854-3a52242640f6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.846864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d987c-2e15-47cf-8854-3a52242640f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.865636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-270d987c-2e15-47cf-8854-3a52242640f6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:25.866873 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-270d987c-2e15-47cf-8854-3a52242640f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:25.884346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9872b9ac4647c4bb98bc8ec82a6ca5', 'request_id': 'req-540b6296-98ca-4d06-b20a-c01e6b88d129'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.897482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] The token cache is being invalidate because project 4c9872b9ac4647c4bb98bc8ec82a6ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:25.897883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.898531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:25.900205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 995/2005] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:25 2022] DELETE /identity/v3/projects/4c9872b9ac4647c4bb98bc8ec82a6ca5 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:25.900985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:25.902366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:25.931824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1011/2006] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:25.932880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d987c-2e15-47cf-8854-3a52242640f6 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:25.934917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-270d987c-2e15-47cf-8854-3a52242640f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:25.947263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d987c-2e15-47cf-8854-3a52242640f6 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:25.952724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:25.953396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.953685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.954000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.954699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:25.956638 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:25.977888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6296-98ca-4d06-b20a-c01e6b88d129 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:25.983633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:25.985049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:25.985563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:25.986070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] PATH_INFO: `/v3/projects/65563e9192834a848521c69c7c80f078` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:25.987672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=65563e9192834a848521c69c7c80f078)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:25.990965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.055745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1012/2007] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:25 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:26.056690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.058618 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:26.122659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65563e9192834a848521c69c7c80f078', 'request_id': 'req-5fe1ee87-9339-4c89-b421-84bf12f70e3d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.126259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af663292-1c09-41b5-a320-1bda0e81017e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.128186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65563e9192834a848521c69c7c80f078', 'request_id': 'req-5fe1ee87-9339-4c89-b421-84bf12f70e3d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.180801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.181681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.181994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.182271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] PATH_INFO: `/v3/users/54760dc7acdd4ce6934b402d2bc7dc47` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.183513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54760dc7acdd4ce6934b402d2bc7dc47)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.185436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.202729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65563e9192834a848521c69c7c80f078', 'request_id': 'req-5fe1ee87-9339-4c89-b421-84bf12f70e3d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.214031 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] The token cache is being invalidate because project 65563e9192834a848521c69c7c80f078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:26.214318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.215513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.221607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 996/2008] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:25 2022] DELETE /identity/v3/projects/65563e9192834a848521c69c7c80f078 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:26.221607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.221607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:26.272858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1ee87-9339-4c89-b421-84bf12f70e3d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.276519 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.277671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.277989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.278269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.280028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:26.291257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.292766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.302541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 997/2009] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:26.303344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.304600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:26.316293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4801ea5-d607-4844-8c2c-b6da73ad5f88 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.322133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc676c-067a-4477-8fa4-6cfa673d0e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.322946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc676c-067a-4477-8fa4-6cfa673d0e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.323076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc676c-067a-4477-8fa4-6cfa673d0e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.323360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc676c-067a-4477-8fa4-6cfa673d0e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.324619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc676c-067a-4477-8fa4-6cfa673d0e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.326891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc676c-067a-4477-8fa4-6cfa673d0e45 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.384151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54760dc7acdd4ce6934b402d2bc7dc47'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.384733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54760dc7acdd4ce6934b402d2bc7dc47'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.479695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 998/2010] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:26 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:26.483509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cd1e7-6cd1-4213-9d69-a135e045bfe8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.483809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cd1e7-6cd1-4213-9d69-a135e045bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.484092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cd1e7-6cd1-4213-9d69-a135e045bfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.502129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1f3cd1e7-6cd1-4213-9d69-a135e045bfe8 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:26.503362 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1f3cd1e7-6cd1-4213-9d69-a135e045bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:26.507540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54760dc7acdd4ce6934b402d2bc7dc47'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.512194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 999/2011] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:26.513081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cd1e7-6cd1-4213-9d69-a135e045bfe8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.514418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1f3cd1e7-6cd1-4213-9d69-a135e045bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:26.515566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54760dc7acdd4ce6934b402d2bc7dc47'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.516215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-3ab89bc1-2f1b-4c53-aa89-411ad8f054c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54760dc7acdd4ce6934b402d2bc7dc47'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:26.526010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cd1e7-6cd1-4213-9d69-a135e045bfe8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.531472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0d5bf-101f-4038-a8a1-158912cbd269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.532129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0d5bf-101f-4038-a8a1-158912cbd269 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.532411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0d5bf-101f-4038-a8a1-158912cbd269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.532718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0d5bf-101f-4038-a8a1-158912cbd269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.533447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0d5bf-101f-4038-a8a1-158912cbd269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.533872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1013/2012] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:26 2022] DELETE /identity/v3/users/54760dc7acdd4ce6934b402d2bc7dc47 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:26.535167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2dfc9-1ca0-4f58-96b8-411a7c72d64f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.535356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0d5bf-101f-4038-a8a1-158912cbd269 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.535717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2dfc9-1ca0-4f58-96b8-411a7c72d64f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.536047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2dfc9-1ca0-4f58-96b8-411a7c72d64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.549606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1000/2013] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:26.551094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0d5bf-101f-4038-a8a1-158912cbd269 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.556530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-cbf2dfc9-1ca0-4f58-96b8-411a7c72d64f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:26.557771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-cbf2dfc9-1ca0-4f58-96b8-411a7c72d64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:26.562130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0d5bf-101f-4038-a8a1-158912cbd269 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.634031 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab767840-0500-47ac-9ff5-08651e39919d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.634776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767840-0500-47ac-9ff5-08651e39919d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.635214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1014/2014] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:26.635541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767840-0500-47ac-9ff5-08651e39919d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.635821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767840-0500-47ac-9ff5-08651e39919d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.637081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab767840-0500-47ac-9ff5-08651e39919d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.639382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab767840-0500-47ac-9ff5-08651e39919d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.640073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2dfc9-1ca0-4f58-96b8-411a7c72d64f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.651061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2dfc9-1ca0-4f58-96b8-411a7c72d64f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.653324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1001/2015] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:26.656732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab767840-0500-47ac-9ff5-08651e39919d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.661437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.662025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.662314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.662613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.663458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:26.663766 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.665570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.672157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab767840-0500-47ac-9ff5-08651e39919d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.677708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95953a3-7ef6-422a-a276-36ec6c16ff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.679284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1015/2016] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:26:26.679690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95953a3-7ef6-422a-a276-36ec6c16ff18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.680942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95953a3-7ef6-422a-a276-36ec6c16ff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.681361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95953a3-7ef6-422a-a276-36ec6c16ff18 admin admin] PATH_INFO: `/v3/projects/d420d393e3b240b98f087d26b7cf738e/users/2a8e46db783b46bb86775c81100b9116/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.686063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.706220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896efa7e-4c4e-4bc2-972b-9950f0b8d4bf admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.706501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95953a3-7ef6-422a-a276-36ec6c16ff18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d420d393e3b240b98f087d26b7cf738e, user_id=2a8e46db783b46bb86775c81100b9116, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.712851 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95953a3-7ef6-422a-a276-36ec6c16ff18 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.713309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935cfa1-d470-4674-be97-93fe4568bd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.714209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935cfa1-d470-4674-be97-93fe4568bd44 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.715669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935cfa1-d470-4674-be97-93fe4568bd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.716433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935cfa1-d470-4674-be97-93fe4568bd44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.717090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935cfa1-d470-4674-be97-93fe4568bd44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.720058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935cfa1-d470-4674-be97-93fe4568bd44 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.860059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1002/2017] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:26 2022] PUT /identity/v3/projects/d420d393e3b240b98f087d26b7cf738e/users/2a8e46db783b46bb86775c81100b9116/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:26.862942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6aa8-1673-48c4-aca8-4d2b9fd8e90f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.863720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6aa8-1673-48c4-aca8-4d2b9fd8e90f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.863720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb6aa8-1673-48c4-aca8-4d2b9fd8e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.866165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1016/2018] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:26 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:26.870202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935cfa1-d470-4674-be97-93fe4568bd44 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.894972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a3fb6aa8-1673-48c4-aca8-4d2b9fd8e90f None None] MFA Rules not processed for user `2a8e46db783b46bb86775c81100b9116`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:26.894972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a3fb6aa8-1673-48c4-aca8-4d2b9fd8e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:26.942822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935cfa1-d470-4674-be97-93fe4568bd44 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:26.950244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8815d2-dcdb-4ca7-86d9-29072210469f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:26.962560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8815d2-dcdb-4ca7-86d9-29072210469f admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:26.962999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8815d2-dcdb-4ca7-86d9-29072210469f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:26.963369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8815d2-dcdb-4ca7-86d9-29072210469f admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:26.966584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8815d2-dcdb-4ca7-86d9-29072210469f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:26.968873 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8815d2-dcdb-4ca7-86d9-29072210469f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:26.988743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1003/2019] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:26.989701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb6aa8-1673-48c4-aca8-4d2b9fd8e90f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:26.991698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a3fb6aa8-1673-48c4-aca8-4d2b9fd8e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.008715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb6aa8-1673-48c4-aca8-4d2b9fd8e90f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.014214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.015120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.015418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.015689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] PATH_INFO: `/v3/projects/e4283617dcb24d1b90259ac807ed087b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.016912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4283617dcb24d1b90259ac807ed087b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.018903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.106039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1017/2020] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:26 2022] POST /identity/v3/users => generated 287 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:27.106039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8815d2-dcdb-4ca7-86d9-29072210469f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.113754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8815d2-dcdb-4ca7-86d9-29072210469f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.118673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb5bef-1a7e-4868-a9ea-4777ad66df5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.119317 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb5bef-1a7e-4868-a9ea-4777ad66df5b admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.119692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb5bef-1a7e-4868-a9ea-4777ad66df5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.119976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb5bef-1a7e-4868-a9ea-4777ad66df5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.120654 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb5bef-1a7e-4868-a9ea-4777ad66df5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.123416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb5bef-1a7e-4868-a9ea-4777ad66df5b admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.154076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4283617dcb24d1b90259ac807ed087b', 'request_id': 'req-71de9edb-c067-42d3-9e20-cc7c32f36ff0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.159667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4283617dcb24d1b90259ac807ed087b', 'request_id': 'req-71de9edb-c067-42d3-9e20-cc7c32f36ff0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.250903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4283617dcb24d1b90259ac807ed087b', 'request_id': 'req-71de9edb-c067-42d3-9e20-cc7c32f36ff0'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.259345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1018/2021] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:27 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:27.259345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb5bef-1a7e-4868-a9ea-4777ad66df5b admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.265180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] The token cache is being invalidate because project e4283617dcb24d1b90259ac807ed087b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:27.265180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.265946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.267686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1004/2022] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:26 2022] DELETE /identity/v3/projects/e4283617dcb24d1b90259ac807ed087b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:27.268604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.270617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.323089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb5bef-1a7e-4868-a9ea-4777ad66df5b admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.328202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.328875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.329157 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.337647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.337647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.337647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.338606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9edb-c067-42d3-9e20-cc7c32f36ff0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.342651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.345988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1019/2023] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:27.346165 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.346165 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.346320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.346394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] PATH_INFO: `/v3/projects/42cb0eaf89b94e948fdbc3862018b5e7` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.346394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42cb0eaf89b94e948fdbc3862018b5e7)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.346886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.347560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.361040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29faecda-e6d3-4537-8a4d-edb21880e4a4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.366647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.367505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.367683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.367972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.368674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.371063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.485051 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42cb0eaf89b94e948fdbc3862018b5e7', 'request_id': 'req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.492006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42cb0eaf89b94e948fdbc3862018b5e7', 'request_id': 'req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.497170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1020/2024] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:27 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:27.501624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.504113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.517174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214c09d-71df-4502-8f6f-2acbe88e2ba4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.564794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42cb0eaf89b94e948fdbc3862018b5e7', 'request_id': 'req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.574999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.575984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.576353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.576690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] PATH_INFO: `/v3/users/a387b4430dd74785b90e87432c4bcfdf` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.577632 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] The token cache is being invalidate because project 42cb0eaf89b94e948fdbc3862018b5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:27.577960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.578175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a387b4430dd74785b90e87432c4bcfdf)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.578695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.583712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1005/2025] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:27 2022] DELETE /identity/v3/projects/42cb0eaf89b94e948fdbc3862018b5e7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:27.583884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.583980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.585297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.601935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f97bfd-fe54-4eaa-b481-5d13e3b7e999 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.607480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.608166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.608457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.608760 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.609481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.611134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.624480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1006/2026] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:27.624480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.626005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.639610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f2504-1e8e-4bd7-806f-7ea6a7058305 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.644307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617a78d-1d65-4055-a8df-78fca5ce8322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.644969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a78d-1d65-4055-a8df-78fca5ce8322 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.645256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a78d-1d65-4055-a8df-78fca5ce8322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.645539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617a78d-1d65-4055-a8df-78fca5ce8322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.646312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617a78d-1d65-4055-a8df-78fca5ce8322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.647922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617a78d-1d65-4055-a8df-78fca5ce8322 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.658618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1007/2027] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:27.659981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213635ac-2214-4dd5-af3f-d93a633b1645 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.660368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213635ac-2214-4dd5-af3f-d93a633b1645 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.660743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213635ac-2214-4dd5-af3f-d93a633b1645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.679739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-213635ac-2214-4dd5-af3f-d93a633b1645 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:27.680956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-213635ac-2214-4dd5-af3f-d93a633b1645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.688761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1008/2028] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:27.689497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213635ac-2214-4dd5-af3f-d93a633b1645 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.699650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213635ac-2214-4dd5-af3f-d93a633b1645 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.704752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.705618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.705934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.706244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] PATH_INFO: `/v3/projects/124fdd0326d04385b1e8cdfe6704145f/users/269972d42449433b8a26c1d364ffc071/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.721296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=124fdd0326d04385b1e8cdfe6704145f, user_id=269972d42449433b8a26c1d364ffc071, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.724000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.789229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a387b4430dd74785b90e87432c4bcfdf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.789843 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a387b4430dd74785b90e87432c4bcfdf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.822529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1009/2029] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:27 2022] PUT /identity/v3/projects/124fdd0326d04385b1e8cdfe6704145f/users/269972d42449433b8a26c1d364ffc071/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:27.832109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:27.832109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.887294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabae54-d28e-4dfc-a5b5-39c3194fc372 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:27.890468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a387b4430dd74785b90e87432c4bcfdf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.892206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e8688-2d55-4c52-a17a-9a32eea45d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:27.893094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e8688-2d55-4c52-a17a-9a32eea45d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.893380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e8688-2d55-4c52-a17a-9a32eea45d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.893660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e8688-2d55-4c52-a17a-9a32eea45d26 admin admin] PATH_INFO: `/v3/projects/f2bb1053c89e4f0996a1a2c7231034a4/users/9b64d09e4c444278a43dd1c9d055e44d/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.898466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a387b4430dd74785b90e87432c4bcfdf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.900762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1a661cbc-590b-4a89-a547-47b1121f8845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a387b4430dd74785b90e87432c4bcfdf'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:27.908357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e8688-2d55-4c52-a17a-9a32eea45d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2bb1053c89e4f0996a1a2c7231034a4, user_id=9b64d09e4c444278a43dd1c9d055e44d, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:27.911090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e8688-2d55-4c52-a17a-9a32eea45d26 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:27.917444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1021/2030] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:27 2022] DELETE /identity/v3/users/a387b4430dd74785b90e87432c4bcfdf => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:27.918879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464041a-5958-4de1-8926-0ee3a6a2aa20 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.919171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464041a-5958-4de1-8926-0ee3a6a2aa20 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.919448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464041a-5958-4de1-8926-0ee3a6a2aa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:27.936602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-7464041a-5958-4de1-8926-0ee3a6a2aa20 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:27.940977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7464041a-5958-4de1-8926-0ee3a6a2aa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:27.996219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1022/2031] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:27.997670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2e34d-e38f-402b-90e5-a03acc4d1fad None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:27.997995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2e34d-e38f-402b-90e5-a03acc4d1fad None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:27.998278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2e34d-e38f-402b-90e5-a03acc4d1fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.024443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-42f2e34d-e38f-402b-90e5-a03acc4d1fad None None] MFA Rules not processed for user `269972d42449433b8a26c1d364ffc071`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:28.025649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-42f2e34d-e38f-402b-90e5-a03acc4d1fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:28.028464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1010/2032] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:27 2022] PUT /identity/v3/projects/f2bb1053c89e4f0996a1a2c7231034a4/users/9b64d09e4c444278a43dd1c9d055e44d/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:28.029156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e8688-2d55-4c52-a17a-9a32eea45d26 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.088941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1023/2033] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:28.090397 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2e34d-e38f-402b-90e5-a03acc4d1fad None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.094647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e8688-2d55-4c52-a17a-9a32eea45d26 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.099651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.100532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.101046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.101339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.102196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:28.102545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.104188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.104345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2e34d-e38f-402b-90e5-a03acc4d1fad None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.109044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df532c8-23ac-4784-a9fc-53661740e240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.109720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df532c8-23ac-4784-a9fc-53661740e240 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.110033 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df532c8-23ac-4784-a9fc-53661740e240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.110310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df532c8-23ac-4784-a9fc-53661740e240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.111050 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df532c8-23ac-4784-a9fc-53661740e240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.112663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df532c8-23ac-4784-a9fc-53661740e240 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.119192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1011/2034] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:26:28.120116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.122571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:28.134514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1024/2035] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:28.135937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df532c8-23ac-4784-a9fc-53661740e240 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.148753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df532c8-23ac-4784-a9fc-53661740e240 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.155029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c1caa-0f79-47c6-9c2d-a37d8caa42f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.155693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c1caa-0f79-47c6-9c2d-a37d8caa42f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.155978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c1caa-0f79-47c6-9c2d-a37d8caa42f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.156330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c1caa-0f79-47c6-9c2d-a37d8caa42f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.156937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c1caa-0f79-47c6-9c2d-a37d8caa42f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.162738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c1caa-0f79-47c6-9c2d-a37d8caa42f5 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.205649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd48ae-acd2-44d2-9b3b-90fb615c81be admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.211231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c8e52-488b-4921-9fb0-9f1926e0aaa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.211878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8e52-488b-4921-9fb0-9f1926e0aaa8 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.212159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8e52-488b-4921-9fb0-9f1926e0aaa8 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.212436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8e52-488b-4921-9fb0-9f1926e0aaa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.220860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c8e52-488b-4921-9fb0-9f1926e0aaa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.222579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c8e52-488b-4921-9fb0-9f1926e0aaa8 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.231791 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1012/2036] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:28.232133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c8e52-488b-4921-9fb0-9f1926e0aaa8 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.242148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c8e52-488b-4921-9fb0-9f1926e0aaa8 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.247725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42885d3f-fc03-493a-8e87-3b47c7a9279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.249256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885d3f-fc03-493a-8e87-3b47c7a9279e admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.249256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885d3f-fc03-493a-8e87-3b47c7a9279e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.249425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42885d3f-fc03-493a-8e87-3b47c7a9279e admin admin] PATH_INFO: `/v3/projects/f2bb1053c89e4f0996a1a2c7231034a4/users/9b64d09e4c444278a43dd1c9d055e44d/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.261579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42885d3f-fc03-493a-8e87-3b47c7a9279e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2bb1053c89e4f0996a1a2c7231034a4, user_id=9b64d09e4c444278a43dd1c9d055e44d, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.261579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42885d3f-fc03-493a-8e87-3b47c7a9279e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.262824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1025/2037] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:28 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:28.264686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c1caa-0f79-47c6-9c2d-a37d8caa42f5 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.266495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1013/2038] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:28 2022] PUT /identity/v3/projects/f2bb1053c89e4f0996a1a2c7231034a4/users/9b64d09e4c444278a43dd1c9d055e44d/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:28.277577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd702b-edd7-4ff7-ba2f-9866d68232f4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.278155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd702b-edd7-4ff7-ba2f-9866d68232f4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.278641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd702b-edd7-4ff7-ba2f-9866d68232f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.312197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e5dd702b-edd7-4ff7-ba2f-9866d68232f4 None None] MFA Rules not processed for user `9b64d09e4c444278a43dd1c9d055e44d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:28.313410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e5dd702b-edd7-4ff7-ba2f-9866d68232f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:28.349133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c1caa-0f79-47c6-9c2d-a37d8caa42f5 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.422958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1014/2039] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:28.422958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd702b-edd7-4ff7-ba2f-9866d68232f4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.422958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e5dd702b-edd7-4ff7-ba2f-9866d68232f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:28.437576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.438303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.438588 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.438863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.439542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.441449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.490633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd702b-edd7-4ff7-ba2f-9866d68232f4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.498423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.499298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.499582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.499861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] PATH_INFO: `/v3/projects/e31a12ca8f6c43149a31520f8fd4065e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.501267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31a12ca8f6c43149a31520f8fd4065e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.503306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.566398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1026/2040] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:28 2022] POST /identity/v3/users => generated 290 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:28.566398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.567636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:28.615969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31a12ca8f6c43149a31520f8fd4065e', 'request_id': 'req-781bd9f2-1f96-4add-9c8c-2a68ced785c1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:28.621805 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31a12ca8f6c43149a31520f8fd4065e', 'request_id': 'req-781bd9f2-1f96-4add-9c8c-2a68ced785c1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:28.624558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee2b6d-1b7d-4ffe-9c54-ae5d30ef88a4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.669437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.670106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.670388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.670662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.672487 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:28.713331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31a12ca8f6c43149a31520f8fd4065e', 'request_id': 'req-781bd9f2-1f96-4add-9c8c-2a68ced785c1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:28.725416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] The token cache is being invalidate because project e31a12ca8f6c43149a31520f8fd4065e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:28.725945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:28.726789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:28.728513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1015/2041] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:28 2022] DELETE /identity/v3/projects/e31a12ca8f6c43149a31520f8fd4065e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:28.729562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.734436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:28.748850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.753773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.771689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1027/2042] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:28.830673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bd9f2-1f96-4add-9c8c-2a68ced785c1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.838377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935504e8-7739-4744-aa2c-fb886acf2905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.841123 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935504e8-7739-4744-aa2c-fb886acf2905 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.841447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935504e8-7739-4744-aa2c-fb886acf2905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.842098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935504e8-7739-4744-aa2c-fb886acf2905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.843040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935504e8-7739-4744-aa2c-fb886acf2905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.844644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935504e8-7739-4744-aa2c-fb886acf2905 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.857895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1016/2043] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:28.861051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.872737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853beb5d-f328-4c26-ad1c-deee545497dd service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.878315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88898d6b-c1ae-406e-8d79-2137919cdc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.879091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898d6b-c1ae-406e-8d79-2137919cdc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.879443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898d6b-c1ae-406e-8d79-2137919cdc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.880229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898d6b-c1ae-406e-8d79-2137919cdc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.880577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88898d6b-c1ae-406e-8d79-2137919cdc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.882321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88898d6b-c1ae-406e-8d79-2137919cdc45 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:28.909932 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935504e8-7739-4744-aa2c-fb886acf2905 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:28.917063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1028/2044] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:28.921288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935504e8-7739-4744-aa2c-fb886acf2905 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:28.927953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536f004-607f-41a4-b8c0-6a92d4cb91a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:28.929099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536f004-607f-41a4-b8c0-6a92d4cb91a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:28.929712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536f004-607f-41a4-b8c0-6a92d4cb91a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:28.930348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536f004-607f-41a4-b8c0-6a92d4cb91a0 admin admin] PATH_INFO: `/v3/projects/0c586081b7ee46d9ad9c973b7b28056a/users/fd2dc5a1361e48b09189f2ae35fff265/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:28.956377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536f004-607f-41a4-b8c0-6a92d4cb91a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c586081b7ee46d9ad9c973b7b28056a, user_id=fd2dc5a1361e48b09189f2ae35fff265, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:28.958778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536f004-607f-41a4-b8c0-6a92d4cb91a0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:29.064741 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1017/2045] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:28 2022] PUT /identity/v3/projects/0c586081b7ee46d9ad9c973b7b28056a/users/fd2dc5a1361e48b09189f2ae35fff265/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:29.068728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b56b3f-7399-4b62-9d16-0b63e0289a88 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.069023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b56b3f-7399-4b62-9d16-0b63e0289a88 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.069302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b56b3f-7399-4b62-9d16-0b63e0289a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.106580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-68b56b3f-7399-4b62-9d16-0b63e0289a88 None None] MFA Rules not processed for user `fd2dc5a1361e48b09189f2ae35fff265`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:29.107713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-68b56b3f-7399-4b62-9d16-0b63e0289a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:29.173183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1029/2046] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:29.177578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536f004-607f-41a4-b8c0-6a92d4cb91a0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:29.245396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536f004-607f-41a4-b8c0-6a92d4cb91a0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:29.250352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13b88d-fe5a-47a8-a70d-eb77a6606a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:29.251475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13b88d-fe5a-47a8-a70d-eb77a6606a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.251475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13b88d-fe5a-47a8-a70d-eb77a6606a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.251661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13b88d-fe5a-47a8-a70d-eb77a6606a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.252229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13b88d-fe5a-47a8-a70d-eb77a6606a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:29.256548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13b88d-fe5a-47a8-a70d-eb77a6606a38 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:29.366244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1018/2047] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:29 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:29.370302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b56b3f-7399-4b62-9d16-0b63e0289a88 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:29.381664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b56b3f-7399-4b62-9d16-0b63e0289a88 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:29.437911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80fe9e-567f-427b-917f-11905ce7b748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:29.438717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80fe9e-567f-427b-917f-11905ce7b748 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.438927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80fe9e-567f-427b-917f-11905ce7b748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.439196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80fe9e-567f-427b-917f-11905ce7b748 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.445073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80fe9e-567f-427b-917f-11905ce7b748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:29.445073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80fe9e-567f-427b-917f-11905ce7b748 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:29.517476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1030/2048] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:29 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:29.524605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13b88d-fe5a-47a8-a70d-eb77a6606a38 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:29.535050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13b88d-fe5a-47a8-a70d-eb77a6606a38 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:29.539994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9e4bd-bccb-4f25-bfe6-d5bc56905dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:29.540659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9e4bd-bccb-4f25-bfe6-d5bc56905dca admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.540946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9e4bd-bccb-4f25-bfe6-d5bc56905dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.541255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9e4bd-bccb-4f25-bfe6-d5bc56905dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.542020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9e4bd-bccb-4f25-bfe6-d5bc56905dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:29.544406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9e4bd-bccb-4f25-bfe6-d5bc56905dca admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:29.555127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1019/2049] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:29.559807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80fe9e-567f-427b-917f-11905ce7b748 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:29.569808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80fe9e-567f-427b-917f-11905ce7b748 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:29.574850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46f0ca-270f-4bd5-9ad7-7458bc25fdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:29.575721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46f0ca-270f-4bd5-9ad7-7458bc25fdda admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.576008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46f0ca-270f-4bd5-9ad7-7458bc25fdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.576287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46f0ca-270f-4bd5-9ad7-7458bc25fdda admin admin] PATH_INFO: `/v3/projects/8971f782c28842b18817e47a48e4b9d7/users/58c9304fbbaa49bd8e0e3db559919747/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.590507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46f0ca-270f-4bd5-9ad7-7458bc25fdda admin admin] RBAC: Authorizing `identity:create_grant(project_id=8971f782c28842b18817e47a48e4b9d7, user_id=58c9304fbbaa49bd8e0e3db559919747, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:29.593237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46f0ca-270f-4bd5-9ad7-7458bc25fdda admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:29.663876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1031/2050] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:29 2022] PUT /identity/v3/projects/8971f782c28842b18817e47a48e4b9d7/users/58c9304fbbaa49bd8e0e3db559919747/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:29.668343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9e4bd-bccb-4f25-bfe6-d5bc56905dca admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:29.679260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9e4bd-bccb-4f25-bfe6-d5bc56905dca admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:29.734973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8692d-ba97-4c9d-aaca-52b58ff65c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:29.735643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8692d-ba97-4c9d-aaca-52b58ff65c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.735924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8692d-ba97-4c9d-aaca-52b58ff65c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.736202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8692d-ba97-4c9d-aaca-52b58ff65c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.736923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8692d-ba97-4c9d-aaca-52b58ff65c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:29.738568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8692d-ba97-4c9d-aaca-52b58ff65c21 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:29.749502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1020/2051] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:29.753590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46f0ca-270f-4bd5-9ad7-7458bc25fdda admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:29.764490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46f0ca-270f-4bd5-9ad7-7458bc25fdda admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:29.769424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:29.770346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.770629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.770903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] PATH_INFO: `/v3/projects/8971f782c28842b18817e47a48e4b9d7/users/58c9304fbbaa49bd8e0e3db559919747/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.772880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8971f782c28842b18817e47a48e4b9d7, user_id=58c9304fbbaa49bd8e0e3db559919747, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:29.775780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:29.786592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1032/2052] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:29 2022] PUT /identity/v3/projects/8971f782c28842b18817e47a48e4b9d7/users/58c9304fbbaa49bd8e0e3db559919747/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:29.802870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614fe4b-dc3f-426f-96b0-b23cd0a61677 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.803161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614fe4b-dc3f-426f-96b0-b23cd0a61677 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.803442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614fe4b-dc3f-426f-96b0-b23cd0a61677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.829748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c614fe4b-dc3f-426f-96b0-b23cd0a61677 None None] MFA Rules not processed for user `58c9304fbbaa49bd8e0e3db559919747`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:29.831011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c614fe4b-dc3f-426f-96b0-b23cd0a61677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:29.900514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1021/2053] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:29.914021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:29.915828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:29.974079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa9058-df66-4665-ad8e-06b03381cea6 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:29.978006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:29.978662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:29.978944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:29.979218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:29.988915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:29.990466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:29.999817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1033/2054] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:30.218202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614fe4b-dc3f-426f-96b0-b23cd0a61677 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:30.227818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614fe4b-dc3f-426f-96b0-b23cd0a61677 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:30.231417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:30.232062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:30.232456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:30.232737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:30.292027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:30.293531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:30.303396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1022/2055] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:31.063106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:31.064568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:31.076436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ff557-b094-42f7-b50a-e8365afb5a92 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:31.080375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620d70f-bbf5-4afe-aaf6-9cfa0a02d535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:31.081012 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620d70f-bbf5-4afe-aaf6-9cfa0a02d535 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:31.081294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620d70f-bbf5-4afe-aaf6-9cfa0a02d535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:31.081567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620d70f-bbf5-4afe-aaf6-9cfa0a02d535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:31.083378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7620d70f-bbf5-4afe-aaf6-9cfa0a02d535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:31.171426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620d70f-bbf5-4afe-aaf6-9cfa0a02d535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:31.172955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620d70f-bbf5-4afe-aaf6-9cfa0a02d535 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:31.182531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1034/2056] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:31.780299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:31.781629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:31.848539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560702e-b9e1-44fa-ba66-4c96e7ac5c1e service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:31.854695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:31.855588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:31.855875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:31.856169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] PATH_INFO: `/v3/users/269972d42449433b8a26c1d364ffc071` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:31.857414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] RBAC: Authorizing `identity:delete_user(user_id=269972d42449433b8a26c1d364ffc071)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:31.859400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:32.063754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269972d42449433b8a26c1d364ffc071'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.064406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269972d42449433b8a26c1d364ffc071'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.065007 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269972d42449433b8a26c1d364ffc071'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.072254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269972d42449433b8a26c1d364ffc071'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.164548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269972d42449433b8a26c1d364ffc071'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.182953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1023/2057] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:31 2022] DELETE /identity/v3/users/269972d42449433b8a26c1d364ffc071 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:32.187958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872fbef-8dc1-495c-a3f1-4a6395da2d0d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:32.188449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872fbef-8dc1-495c-a3f1-4a6395da2d0d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:32.188540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872fbef-8dc1-495c-a3f1-4a6395da2d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:32.206774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-7872fbef-8dc1-495c-a3f1-4a6395da2d0d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:32.207980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7872fbef-8dc1-495c-a3f1-4a6395da2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:32.267094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1035/2058] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:32.279074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:32.279236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:32.344217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b4cfb-e122-41e5-863a-f358a072901e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:32.349305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:32.350010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:32.350300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:32.350579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:32.361264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:32.362737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:32.373447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1024/2059] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:32.598143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872fbef-8dc1-495c-a3f1-4a6395da2d0d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:32.600036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7872fbef-8dc1-495c-a3f1-4a6395da2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:32.618704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872fbef-8dc1-495c-a3f1-4a6395da2d0d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:32.624892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:32.626484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:32.626885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:32.627266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] PATH_INFO: `/v3/users/fe466c6e49404e52b44cc49ef1475140` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:32.628632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe466c6e49404e52b44cc49ef1475140)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:32.631095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:32.822063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe466c6e49404e52b44cc49ef1475140'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.822830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe466c6e49404e52b44cc49ef1475140'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.827216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:32.828776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:32.898145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01a54-f354-44fa-8b17-b4dac6848307 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:32.901939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe466c6e49404e52b44cc49ef1475140'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.905534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:32.906259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:32.906567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:32.906841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:32.909617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:32.913945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe466c6e49404e52b44cc49ef1475140'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.914930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe466c6e49404e52b44cc49ef1475140'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:32.940655 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1036/2060] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:32 2022] DELETE /identity/v3/users/fe466c6e49404e52b44cc49ef1475140 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:32.978090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:32.979549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:32.991301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1025/2061] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:33.013972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.018455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:33.094748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71d62b-7324-4b10-b08f-961162ddafda admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.100227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.100750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.101031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.101302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] PATH_INFO: `/v3/projects/124fdd0326d04385b1e8cdfe6704145f` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.102818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=124fdd0326d04385b1e8cdfe6704145f)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.104858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.218398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.219976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:33.234968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c24fe-d20b-428a-a406-1988c6e17a83 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.239635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124fdd0326d04385b1e8cdfe6704145f', 'request_id': 'req-f4775bec-2229-4d6e-879d-6a6e75758c6c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.255804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124fdd0326d04385b1e8cdfe6704145f', 'request_id': 'req-f4775bec-2229-4d6e-879d-6a6e75758c6c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.294091 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.294958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.295230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.295503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] PATH_INFO: `/v3/projects/6dc12e51fbc2450ab2d7868463bff90e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.296694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dc12e51fbc2450ab2d7868463bff90e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.298690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.336048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124fdd0326d04385b1e8cdfe6704145f', 'request_id': 'req-f4775bec-2229-4d6e-879d-6a6e75758c6c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.347535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] The token cache is being invalidate because project 124fdd0326d04385b1e8cdfe6704145f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:33.347822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.348455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f4775bec-2229-4d6e-879d-6a6e75758c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.350056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1037/2062] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:33 2022] DELETE /identity/v3/projects/124fdd0326d04385b1e8cdfe6704145f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:33.356703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878570-0be3-4d69-b2c4-c937aec9a11b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.356952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878570-0be3-4d69-b2c4-c937aec9a11b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.357239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878570-0be3-4d69-b2c4-c937aec9a11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.375629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-45878570-0be3-4d69-b2c4-c937aec9a11b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:33.376831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-45878570-0be3-4d69-b2c4-c937aec9a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:33.384967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1038/2063] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:33.392204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dfb4-d98d-4d73-a94e-63ebe0c1a841 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.392493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dfb4-d98d-4d73-a94e-63ebe0c1a841 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.392775 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9dfb4-d98d-4d73-a94e-63ebe0c1a841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.411490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-21e9dfb4-d98d-4d73-a94e-63ebe0c1a841 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:33.412688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-21e9dfb4-d98d-4d73-a94e-63ebe0c1a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:33.421148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1039/2064] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:33.425242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9dfb4-d98d-4d73-a94e-63ebe0c1a841 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.434530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9dfb4-d98d-4d73-a94e-63ebe0c1a841 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.439252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.439674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.439959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.440234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.440942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:33.441275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.442895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.454571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1040/2065] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:33.458482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.468084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f31aa0-e884-430d-9f34-cb0c823442bc admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.472772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b332f-21f8-4553-9413-5431b9b3a7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.473416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b332f-21f8-4553-9413-5431b9b3a7c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.473703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b332f-21f8-4553-9413-5431b9b3a7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.474017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b332f-21f8-4553-9413-5431b9b3a7c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.474699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b332f-21f8-4553-9413-5431b9b3a7c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.476593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b332f-21f8-4553-9413-5431b9b3a7c0 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.497677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dc12e51fbc2450ab2d7868463bff90e', 'request_id': 'req-10828134-4fb7-4ff6-86aa-cc2243452af7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.508175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dc12e51fbc2450ab2d7868463bff90e', 'request_id': 'req-10828134-4fb7-4ff6-86aa-cc2243452af7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.583481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1041/2066] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:33 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:33.587553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b332f-21f8-4553-9413-5431b9b3a7c0 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.642801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dc12e51fbc2450ab2d7868463bff90e', 'request_id': 'req-10828134-4fb7-4ff6-86aa-cc2243452af7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.647476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b332f-21f8-4553-9413-5431b9b3a7c0 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.652837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.653526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.653836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.654150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.654839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.655719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] The token cache is being invalidate because project 6dc12e51fbc2450ab2d7868463bff90e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:33.656012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.656680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.656788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.658472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1026/2067] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:33 2022] DELETE /identity/v3/projects/6dc12e51fbc2450ab2d7868463bff90e => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:33.662139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.663557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:33.726811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10828134-4fb7-4ff6-86aa-cc2243452af7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.732041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.732968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.733242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.733525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] PATH_INFO: `/v3/users/30c0f366d1e445ad91d56b1725d3b093` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.734774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c0f366d1e445ad91d56b1725d3b093)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.736733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.764738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1042/2068] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:33 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:33.768625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.770373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:33.786020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f0509-9178-47d0-b580-63baf385abe2 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.790837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de539-d41f-45b4-9a76-6bd5dc45f176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.791528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de539-d41f-45b4-9a76-6bd5dc45f176 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.791867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de539-d41f-45b4-9a76-6bd5dc45f176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.792148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9de539-d41f-45b4-9a76-6bd5dc45f176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.792915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9de539-d41f-45b4-9a76-6bd5dc45f176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.795181 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9de539-d41f-45b4-9a76-6bd5dc45f176 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.805053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1043/2069] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:33.808904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de539-d41f-45b4-9a76-6bd5dc45f176 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.818059 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9de539-d41f-45b4-9a76-6bd5dc45f176 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.822887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd81ac-d3a2-4c93-8f06-b749a47cf114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.823504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd81ac-d3a2-4c93-8f06-b749a47cf114 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.823791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd81ac-d3a2-4c93-8f06-b749a47cf114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.824567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd81ac-d3a2-4c93-8f06-b749a47cf114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.825260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd81ac-d3a2-4c93-8f06-b749a47cf114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.826852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd81ac-d3a2-4c93-8f06-b749a47cf114 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.836943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1044/2070] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:33.841424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd81ac-d3a2-4c93-8f06-b749a47cf114 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:33.850821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd81ac-d3a2-4c93-8f06-b749a47cf114 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:33.855491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad0355-dd08-468c-a478-bad81ce6d541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:33.856335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad0355-dd08-468c-a478-bad81ce6d541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:33.856620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad0355-dd08-468c-a478-bad81ce6d541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:33.856896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad0355-dd08-468c-a478-bad81ce6d541 admin admin] PATH_INFO: `/v3/projects/8a768d97ae0944f19d69318e32c9948d/users/32afde1423bf4983a6070c9203510d7e/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:33.870937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad0355-dd08-468c-a478-bad81ce6d541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a768d97ae0944f19d69318e32c9948d, user_id=32afde1423bf4983a6070c9203510d7e, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:33.873526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad0355-dd08-468c-a478-bad81ce6d541 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:33.968209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c0f366d1e445ad91d56b1725d3b093'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.969090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c0f366d1e445ad91d56b1725d3b093'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.969905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c0f366d1e445ad91d56b1725d3b093'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.977226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c0f366d1e445ad91d56b1725d3b093'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:33.993959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1045/2071] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:33 2022] PUT /identity/v3/projects/8a768d97ae0944f19d69318e32c9948d/users/32afde1423bf4983a6070c9203510d7e/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:34.007640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257ed8-a31d-4378-ac9f-ac90bef9d2d1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:34.007937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257ed8-a31d-4378-ac9f-ac90bef9d2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:34.008220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257ed8-a31d-4378-ac9f-ac90bef9d2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:34.034798 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-4a257ed8-a31d-4378-ac9f-ac90bef9d2d1 None None] MFA Rules not processed for user `32afde1423bf4983a6070c9203510d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:34.035995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4a257ed8-a31d-4378-ac9f-ac90bef9d2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:34.050544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c0f366d1e445ad91d56b1725d3b093'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:34.067831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1027/2072] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:33 2022] DELETE /identity/v3/users/30c0f366d1e445ad91d56b1725d3b093 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:34.099857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1046/2073] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:34.111409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:34.115534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:34.174222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2644ab-9bf7-4229-b94d-d16147d87581 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:34.177858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e383df-5ef2-4159-bd93-e8bd55adc1d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:34.178508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e383df-5ef2-4159-bd93-e8bd55adc1d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:34.178790 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e383df-5ef2-4159-bd93-e8bd55adc1d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:34.179070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e383df-5ef2-4159-bd93-e8bd55adc1d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:34.235257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e383df-5ef2-4159-bd93-e8bd55adc1d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:34.236732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e383df-5ef2-4159-bd93-e8bd55adc1d6 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:34.245664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1028/2074] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:34.338272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257ed8-a31d-4378-ac9f-ac90bef9d2d1 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:34.339906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4a257ed8-a31d-4378-ac9f-ac90bef9d2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:34.409936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257ed8-a31d-4378-ac9f-ac90bef9d2d1 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:34.464162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:34.465049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:34.465333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:34.465611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] PATH_INFO: `/v3/projects/6183251fc8a54a829b220a38088df86d` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:34.466829 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6183251fc8a54a829b220a38088df86d)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:34.468745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:34.633471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6183251fc8a54a829b220a38088df86d', 'request_id': 'req-9687cf4d-b020-4ff6-ac54-2774c328ca9e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:34.639152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6183251fc8a54a829b220a38088df86d', 'request_id': 'req-9687cf4d-b020-4ff6-ac54-2774c328ca9e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:34.719452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6183251fc8a54a829b220a38088df86d', 'request_id': 'req-9687cf4d-b020-4ff6-ac54-2774c328ca9e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:34.733750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] The token cache is being invalidate because project 6183251fc8a54a829b220a38088df86d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:34.733750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:34.733750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-9687cf4d-b020-4ff6-ac54-2774c328ca9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:34.734421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1047/2075] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:34 2022] DELETE /identity/v3/projects/6183251fc8a54a829b220a38088df86d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:34.739472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e67853-a28b-4026-a0c3-3333315d60aa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:34.739764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e67853-a28b-4026-a0c3-3333315d60aa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:34.740045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e67853-a28b-4026-a0c3-3333315d60aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:34.757623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e9e67853-a28b-4026-a0c3-3333315d60aa None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:34.758900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e9e67853-a28b-4026-a0c3-3333315d60aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:34.835538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1029/2076] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:34.843571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114fa6e-96a8-4f71-b313-8e28b6f32987 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:34.843862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114fa6e-96a8-4f71-b313-8e28b6f32987 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:34.844139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114fa6e-96a8-4f71-b313-8e28b6f32987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:34.862671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6114fa6e-96a8-4f71-b313-8e28b6f32987 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:34.863915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6114fa6e-96a8-4f71-b313-8e28b6f32987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:34.871993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1048/2077] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:34.882218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e67853-a28b-4026-a0c3-3333315d60aa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:34.891754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e67853-a28b-4026-a0c3-3333315d60aa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:34.896550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:34.897202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:34.897489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:34.897820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:34.898553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:34.898862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:34.900470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:34.911991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1030/2078] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:26:34.916647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114fa6e-96a8-4f71-b313-8e28b6f32987 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:34.926428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114fa6e-96a8-4f71-b313-8e28b6f32987 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:34.931211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36501e53-c30c-41bc-8242-ca4953504827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:34.931895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501e53-c30c-41bc-8242-ca4953504827 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:34.932180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501e53-c30c-41bc-8242-ca4953504827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:34.932485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501e53-c30c-41bc-8242-ca4953504827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:34.933201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36501e53-c30c-41bc-8242-ca4953504827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:34.935779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36501e53-c30c-41bc-8242-ca4953504827 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:35.038158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1049/2079] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:34 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:35.039351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:35.049873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73da335-6597-4705-a7b8-5237f6403ae7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:35.106591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1b1eb-2f6b-4042-82c9-05742cff7765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:35.107260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1b1eb-2f6b-4042-82c9-05742cff7765 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.107548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1b1eb-2f6b-4042-82c9-05742cff7765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.107833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1b1eb-2f6b-4042-82c9-05742cff7765 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.109026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1b1eb-2f6b-4042-82c9-05742cff7765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:35.110464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1b1eb-2f6b-4042-82c9-05742cff7765 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:35.206582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1031/2080] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:35 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:35.210707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36501e53-c30c-41bc-8242-ca4953504827 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:35.222270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36501e53-c30c-41bc-8242-ca4953504827 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:35.228051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c5ed0-1ade-478a-9372-e5da674b3d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:35.228813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c5ed0-1ade-478a-9372-e5da674b3d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.229236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c5ed0-1ade-478a-9372-e5da674b3d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.229660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c5ed0-1ade-478a-9372-e5da674b3d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.230627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c5ed0-1ade-478a-9372-e5da674b3d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:35.233087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c5ed0-1ade-478a-9372-e5da674b3d7b admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:35.244855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1050/2081] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:35.249043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1b1eb-2f6b-4042-82c9-05742cff7765 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:35.258703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1b1eb-2f6b-4042-82c9-05742cff7765 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:35.263265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cef93-0299-44f4-9fd8-67d9c39216ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:35.263906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cef93-0299-44f4-9fd8-67d9c39216ea admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.264184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cef93-0299-44f4-9fd8-67d9c39216ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.264459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cef93-0299-44f4-9fd8-67d9c39216ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.265177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cef93-0299-44f4-9fd8-67d9c39216ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:35.266785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cef93-0299-44f4-9fd8-67d9c39216ea admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:35.276267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1032/2082] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:35.280571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c5ed0-1ade-478a-9372-e5da674b3d7b admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:35.290248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c5ed0-1ade-478a-9372-e5da674b3d7b admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:35.295649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678507-8c11-4025-b51f-5a528b467ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:35.296514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678507-8c11-4025-b51f-5a528b467ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.296793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678507-8c11-4025-b51f-5a528b467ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.297067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678507-8c11-4025-b51f-5a528b467ea8 admin admin] PATH_INFO: `/v3/projects/ae10ab699bed431b808aafa89d940578/users/1bbf4e1b0bc14b038c2ac580d61f673f/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.312398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678507-8c11-4025-b51f-5a528b467ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae10ab699bed431b808aafa89d940578, user_id=1bbf4e1b0bc14b038c2ac580d61f673f, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:35.315113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7678507-8c11-4025-b51f-5a528b467ea8 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:35.460152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1051/2083] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:35 2022] PUT /identity/v3/projects/ae10ab699bed431b808aafa89d940578/users/1bbf4e1b0bc14b038c2ac580d61f673f/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:35.470944 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e342cad-eb18-49cf-9961-a180ad4bf7ea None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.471239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e342cad-eb18-49cf-9961-a180ad4bf7ea None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.471520 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e342cad-eb18-49cf-9961-a180ad4bf7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.497720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6e342cad-eb18-49cf-9961-a180ad4bf7ea None None] MFA Rules not processed for user `1bbf4e1b0bc14b038c2ac580d61f673f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:35.499133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6e342cad-eb18-49cf-9961-a180ad4bf7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:35.572456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1033/2084] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:35.577908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678507-8c11-4025-b51f-5a528b467ea8 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:35.637325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678507-8c11-4025-b51f-5a528b467ea8 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:35.642003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a9ded-db1f-43b0-a617-7c6149031f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:35.642658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a9ded-db1f-43b0-a617-7c6149031f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.642942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a9ded-db1f-43b0-a617-7c6149031f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.643223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a9ded-db1f-43b0-a617-7c6149031f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.643919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a9ded-db1f-43b0-a617-7c6149031f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:35.645840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a9ded-db1f-43b0-a617-7c6149031f49 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:35.754948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1052/2085] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:35.758943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e342cad-eb18-49cf-9961-a180ad4bf7ea None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:35.770214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e342cad-eb18-49cf-9961-a180ad4bf7ea None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:35.825738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfe2e3-2033-470e-8fee-2ccccd703369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:35.826445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfe2e3-2033-470e-8fee-2ccccd703369 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.826727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfe2e3-2033-470e-8fee-2ccccd703369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.827006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfe2e3-2033-470e-8fee-2ccccd703369 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.827709 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfe2e3-2033-470e-8fee-2ccccd703369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:35.829620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfe2e3-2033-470e-8fee-2ccccd703369 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:35.922428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1034/2086] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:35 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:35.923687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a9ded-db1f-43b0-a617-7c6149031f49 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:35.935148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a9ded-db1f-43b0-a617-7c6149031f49 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:35.941053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebd58e-7fb7-4733-8436-bbd9e8593d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:35.941897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd58e-7fb7-4733-8436-bbd9e8593d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.942360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd58e-7fb7-4733-8436-bbd9e8593d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.942666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd58e-7fb7-4733-8436-bbd9e8593d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.943521 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebd58e-7fb7-4733-8436-bbd9e8593d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:35.945238 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebd58e-7fb7-4733-8436-bbd9e8593d82 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:35.958025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1053/2087] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:35.961933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfe2e3-2033-470e-8fee-2ccccd703369 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:35.972139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfe2e3-2033-470e-8fee-2ccccd703369 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:35.976810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6035e-abe6-4084-8a24-ad79cfcdcfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:35.977681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6035e-abe6-4084-8a24-ad79cfcdcfdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:35.978004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6035e-abe6-4084-8a24-ad79cfcdcfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:35.978284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6035e-abe6-4084-8a24-ad79cfcdcfdc admin admin] PATH_INFO: `/v3/projects/0b04f57c5c3e4702b0d175d72596261c/users/cc236d63727d40368dd663a3544324f9/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:35.992674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6035e-abe6-4084-8a24-ad79cfcdcfdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b04f57c5c3e4702b0d175d72596261c, user_id=cc236d63727d40368dd663a3544324f9, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:35.995426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6035e-abe6-4084-8a24-ad79cfcdcfdc admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:36.113450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1035/2088] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:35 2022] PUT /identity/v3/projects/0b04f57c5c3e4702b0d175d72596261c/users/cc236d63727d40368dd663a3544324f9/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:36.122246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebd58e-7fb7-4733-8436-bbd9e8593d82 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:36.132071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebd58e-7fb7-4733-8436-bbd9e8593d82 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:36.185605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6765d34-962a-4c2b-a18a-263496bf871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:36.186307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6765d34-962a-4c2b-a18a-263496bf871e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:36.186589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6765d34-962a-4c2b-a18a-263496bf871e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:36.186864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6765d34-962a-4c2b-a18a-263496bf871e admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:36.187607 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6765d34-962a-4c2b-a18a-263496bf871e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:36.189201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6765d34-962a-4c2b-a18a-263496bf871e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:36.198931 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1054/2089] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:36.203057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6035e-abe6-4084-8a24-ad79cfcdcfdc admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:36.213248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6035e-abe6-4084-8a24-ad79cfcdcfdc admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:36.217688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:36.218585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:36.218864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:36.219754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] PATH_INFO: `/v3/projects/0b04f57c5c3e4702b0d175d72596261c/users/cc236d63727d40368dd663a3544324f9/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:36.221833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b04f57c5c3e4702b0d175d72596261c, user_id=cc236d63727d40368dd663a3544324f9, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:36.224644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:36.235223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1036/2090] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:36 2022] PUT /identity/v3/projects/0b04f57c5c3e4702b0d175d72596261c/users/cc236d63727d40368dd663a3544324f9/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:36.246658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2275c-823d-4565-8cd7-f9cdd140c4f0 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:36.246953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2275c-823d-4565-8cd7-f9cdd140c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:36.247233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2275c-823d-4565-8cd7-f9cdd140c4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:36.272237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-8dd2275c-823d-4565-8cd7-f9cdd140c4f0 None None] MFA Rules not processed for user `cc236d63727d40368dd663a3544324f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:36.273433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8dd2275c-823d-4565-8cd7-f9cdd140c4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:36.339105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1055/2091] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:36.349697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:36.351418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:36.406953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe3bc3-9989-444c-923a-12f84c33e5a8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:36.410619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25301-6666-4f3a-bac7-db12d1511bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:36.411263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25301-6666-4f3a-bac7-db12d1511bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:36.411548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25301-6666-4f3a-bac7-db12d1511bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:36.411825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25301-6666-4f3a-bac7-db12d1511bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:36.467599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f25301-6666-4f3a-bac7-db12d1511bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:36.469041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f25301-6666-4f3a-bac7-db12d1511bc8 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:36.478240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1037/2092] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:40.567794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2275c-823d-4565-8cd7-f9cdd140c4f0 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:40.569235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8dd2275c-823d-4565-8cd7-f9cdd140c4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:40.633434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2275c-823d-4565-8cd7-f9cdd140c4f0 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:40.638670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:40.639539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:40.639823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:40.640101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] PATH_INFO: `/v3/users/d2f18afce16e43df97efa0f18c47fb2b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:40.641340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f18afce16e43df97efa0f18c47fb2b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:40.643313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:40.835879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f18afce16e43df97efa0f18c47fb2b'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:40.836458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f18afce16e43df97efa0f18c47fb2b'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:40.905609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f18afce16e43df97efa0f18c47fb2b'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:40.913405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f18afce16e43df97efa0f18c47fb2b'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:40.915862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f18afce16e43df97efa0f18c47fb2b'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:40.931248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1056/2093] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:40 2022] DELETE /identity/v3/users/d2f18afce16e43df97efa0f18c47fb2b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:40.935914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1e058-722d-4696-877c-19d8afd23efe None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:40.936435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1e058-722d-4696-877c-19d8afd23efe None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:40.936527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1e058-722d-4696-877c-19d8afd23efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:40.954130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-63f1e058-722d-4696-877c-19d8afd23efe None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:40.955330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-63f1e058-722d-4696-877c-19d8afd23efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:41.015668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1038/2094] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:41.025282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:41.026768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:41.082031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15981-2095-47e7-9453-594b5cae644e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:41.085695 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679ff88-3544-4d3f-abce-749eacb05598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:41.086416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ff88-3544-4d3f-abce-749eacb05598 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:41.086646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ff88-3544-4d3f-abce-749eacb05598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:41.086922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ff88-3544-4d3f-abce-749eacb05598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:41.096120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679ff88-3544-4d3f-abce-749eacb05598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:41.097736 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679ff88-3544-4d3f-abce-749eacb05598 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:41.107547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1057/2095] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:41.561225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1e058-722d-4696-877c-19d8afd23efe None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:41.562737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-63f1e058-722d-4696-877c-19d8afd23efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:41.574733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1e058-722d-4696-877c-19d8afd23efe None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:41.579380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:41.580335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:41.580681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:41.586953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] PATH_INFO: `/v3/projects/564789086e0f4c91a2159872edba8e91` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:41.586953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=564789086e0f4c91a2159872edba8e91)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:41.586953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:41.758861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564789086e0f4c91a2159872edba8e91', 'request_id': 'req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:41.764137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564789086e0f4c91a2159872edba8e91', 'request_id': 'req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:41.870055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564789086e0f4c91a2159872edba8e91', 'request_id': 'req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:41.881727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] The token cache is being invalidate because project 564789086e0f4c91a2159872edba8e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:41.882049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:41.882693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:41.884323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1039/2096] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:41 2022] DELETE /identity/v3/projects/564789086e0f4c91a2159872edba8e91 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:41.888423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679ff88-3544-4d3f-abce-749eacb05598 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:41.900205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679ff88-3544-4d3f-abce-749eacb05598 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:41.968660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:41.969547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:41.969862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:41.970143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] PATH_INFO: `/v3/users/2a8e46db783b46bb86775c81100b9116` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:41.971630 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8e46db783b46bb86775c81100b9116)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:41.973967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:42.150657 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8e46db783b46bb86775c81100b9116'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:42.151278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8e46db783b46bb86775c81100b9116'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:42.256549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8e46db783b46bb86775c81100b9116'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:42.263870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8e46db783b46bb86775c81100b9116'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:42.264577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8e46db783b46bb86775c81100b9116'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:42.282544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1058/2097] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:41 2022] DELETE /identity/v3/users/2a8e46db783b46bb86775c81100b9116 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:43.069083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:43.077037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:43.083688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f20381-72ad-4cf5-95c1-41e37ad1e6ff admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:43.167618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:43.169030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:43.169310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:43.169581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] PATH_INFO: `/v3/projects/d420d393e3b240b98f087d26b7cf738e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:43.170795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d420d393e3b240b98f087d26b7cf738e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:43.172692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:43.255082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:43.256802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:43.292555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd420d393e3b240b98f087d26b7cf738e', 'request_id': 'req-fe02fba1-45db-416f-bdc8-4650a26234f7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.299479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd420d393e3b240b98f087d26b7cf738e', 'request_id': 'req-fe02fba1-45db-416f-bdc8-4650a26234f7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.315636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5a2e5-a55e-44dd-8aba-eb5f8e1b4310 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:43.365526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:43.366203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:43.366488 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:43.366826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:43.368611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:43.405213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd420d393e3b240b98f087d26b7cf738e', 'request_id': 'req-fe02fba1-45db-416f-bdc8-4650a26234f7'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.416953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] The token cache is being invalidate because project d420d393e3b240b98f087d26b7cf738e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:43.417240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.417954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.420014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1040/2098] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:43 2022] DELETE /identity/v3/projects/d420d393e3b240b98f087d26b7cf738e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:43.424117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:43.425477 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:43.430697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:43.432139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:43.442338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1059/2099] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:43.492877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02fba1-45db-416f-bdc8-4650a26234f7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:43.497740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:43.498662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:43.498936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:43.499212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] PATH_INFO: `/v3/users/9b64d09e4c444278a43dd1c9d055e44d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:43.500426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b64d09e4c444278a43dd1c9d055e44d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:43.502512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:43.785857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b64d09e4c444278a43dd1c9d055e44d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.786581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b64d09e4c444278a43dd1c9d055e44d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.787518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b64d09e4c444278a43dd1c9d055e44d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.796063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b64d09e4c444278a43dd1c9d055e44d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.879720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b64d09e4c444278a43dd1c9d055e44d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:43.899253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1041/2100] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:43 2022] DELETE /identity/v3/users/9b64d09e4c444278a43dd1c9d055e44d => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:44.070182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:44.071608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:44.084926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae38f18-09c0-494f-b447-606d2e9cd214 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:44.140560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:44.141752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:44.142827 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:44.142827 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] PATH_INFO: `/v3/users/fd2dc5a1361e48b09189f2ae35fff265` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:44.143900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2dc5a1361e48b09189f2ae35fff265)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:44.146112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:44.332056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2dc5a1361e48b09189f2ae35fff265'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.332956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2dc5a1361e48b09189f2ae35fff265'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.383913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:44.385361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:44.400061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1cc97-db19-438d-a3a0-a8b84cfa3451 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:44.405359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2dc5a1361e48b09189f2ae35fff265'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.405505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:44.406370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:44.406657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:44.406933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] PATH_INFO: `/v3/projects/f2bb1053c89e4f0996a1a2c7231034a4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:44.408601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2bb1053c89e4f0996a1a2c7231034a4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:44.415505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:44.417172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2dc5a1361e48b09189f2ae35fff265'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.418464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-164bbf1e-6ddf-4a6b-8b13-9283a718e39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2dc5a1361e48b09189f2ae35fff265'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.438117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1060/2101] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:44 2022] DELETE /identity/v3/users/fd2dc5a1361e48b09189f2ae35fff265 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:44.444148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e5d8-17f7-4230-adce-50ac95ca0e8d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:44.444438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e5d8-17f7-4230-adce-50ac95ca0e8d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:44.444718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3e5d8-17f7-4230-adce-50ac95ca0e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:44.462776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bbc3e5d8-17f7-4230-adce-50ac95ca0e8d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:44.464077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bbc3e5d8-17f7-4230-adce-50ac95ca0e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:44.526309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1061/2102] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:44.537059 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3e5d8-17f7-4230-adce-50ac95ca0e8d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:44.538579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bbc3e5d8-17f7-4230-adce-50ac95ca0e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:44.557613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2bb1053c89e4f0996a1a2c7231034a4', 'request_id': 'req-2596b412-f17c-4319-82a1-c80280696bcd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.563410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2bb1053c89e4f0996a1a2c7231034a4', 'request_id': 'req-2596b412-f17c-4319-82a1-c80280696bcd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.599192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3e5d8-17f7-4230-adce-50ac95ca0e8d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:44.650274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cb409-3464-4800-9d0c-8d9a3fda4e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:44.650933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb409-3464-4800-9d0c-8d9a3fda4e13 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:44.651222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb409-3464-4800-9d0c-8d9a3fda4e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:44.651500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb409-3464-4800-9d0c-8d9a3fda4e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:44.684851 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2bb1053c89e4f0996a1a2c7231034a4', 'request_id': 'req-2596b412-f17c-4319-82a1-c80280696bcd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.707907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] The token cache is being invalidate because project f2bb1053c89e4f0996a1a2c7231034a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:44.708366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.709250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2596b412-f17c-4319-82a1-c80280696bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:44.710958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cb409-3464-4800-9d0c-8d9a3fda4e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:44.712687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cb409-3464-4800-9d0c-8d9a3fda4e13 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:44.714767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1042/2103] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:44 2022] DELETE /identity/v3/projects/f2bb1053c89e4f0996a1a2c7231034a4 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:44.716606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a7045-b92a-4919-87cd-e183ea47308d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:44.717056 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a7045-b92a-4919-87cd-e183ea47308d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:44.717391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a7045-b92a-4919-87cd-e183ea47308d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:44.725293 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1062/2104] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:44.739816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-aa5a7045-b92a-4919-87cd-e183ea47308d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:44.741049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-aa5a7045-b92a-4919-87cd-e183ea47308d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:44.750829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1043/2105] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:44.759471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7cfd-bc94-4ab2-af84-e259d99ac0a4 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:44.759762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7cfd-bc94-4ab2-af84-e259d99ac0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:44.760044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7cfd-bc94-4ab2-af84-e259d99ac0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:44.778832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-5c1b7cfd-bc94-4ab2-af84-e259d99ac0a4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:44.780058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5c1b7cfd-bc94-4ab2-af84-e259d99ac0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:44.788751 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1063/2106] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:44.792841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a7045-b92a-4919-87cd-e183ea47308d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:44.802589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a7045-b92a-4919-87cd-e183ea47308d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:44.808273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:44.809423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:44.809827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:44.810313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:44.810771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:44.811080 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:44.812882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:44.840203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1044/2107] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:26:44.841680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b7cfd-bc94-4ab2-af84-e259d99ac0a4 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:44.851490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b7cfd-bc94-4ab2-af84-e259d99ac0a4 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:44.858031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eff777-2582-43ec-b7fe-0799f3cce09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:44.858853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eff777-2582-43ec-b7fe-0799f3cce09b admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:44.859207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eff777-2582-43ec-b7fe-0799f3cce09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:44.859494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eff777-2582-43ec-b7fe-0799f3cce09b admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:44.860249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eff777-2582-43ec-b7fe-0799f3cce09b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:44.862330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eff777-2582-43ec-b7fe-0799f3cce09b admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:44.952057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1064/2108] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:44 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:44.955957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:44.967178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43fe55-784f-440b-8823-4e2f388b3ee6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:45.034270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805c3c3-07e5-4860-a80b-438745424c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:45.035225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805c3c3-07e5-4860-a80b-438745424c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:45.035521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805c3c3-07e5-4860-a80b-438745424c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:45.035803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805c3c3-07e5-4860-a80b-438745424c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:45.036515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805c3c3-07e5-4860-a80b-438745424c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:45.039447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805c3c3-07e5-4860-a80b-438745424c28 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:45.113834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1045/2109] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:44 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:45.117958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eff777-2582-43ec-b7fe-0799f3cce09b admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:45.129432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eff777-2582-43ec-b7fe-0799f3cce09b admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:45.134393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7c76d-62b3-4914-ac99-fe0540142dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:45.137570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7c76d-62b3-4914-ac99-fe0540142dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:45.137570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7c76d-62b3-4914-ac99-fe0540142dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:45.137570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7c76d-62b3-4914-ac99-fe0540142dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:45.137570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7c76d-62b3-4914-ac99-fe0540142dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:45.138007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7c76d-62b3-4914-ac99-fe0540142dbf admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:45.147904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1065/2110] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:45.152110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805c3c3-07e5-4860-a80b-438745424c28 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:45.162549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805c3c3-07e5-4860-a80b-438745424c28 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:45.167598 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:45.168884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:45.169271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:45.169620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:45.170468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:45.172184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:45.183038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1046/2111] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:45.187179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7c76d-62b3-4914-ac99-fe0540142dbf admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:45.199742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:45.202585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7c76d-62b3-4914-ac99-fe0540142dbf admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:45.205582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:45.215572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a2556-3471-43c3-9bde-e11fc212301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:45.215972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2556-3471-43c3-9bde-e11fc212301a admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:45.216251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2556-3471-43c3-9bde-e11fc212301a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:45.216520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2556-3471-43c3-9bde-e11fc212301a admin admin] PATH_INFO: `/v3/projects/3ea1460c53954c6a915fc638a2690cbf/users/1cdee89f8a8040ca944458d3fc6f18eb/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:45.219494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93229947-879d-42c1-a07e-af472cc7f550 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:45.228195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:45.228195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:45.228195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:45.228195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] PATH_INFO: `/v3/projects/0c586081b7ee46d9ad9c973b7b28056a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:45.228843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c586081b7ee46d9ad9c973b7b28056a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:45.232231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:45.237811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a2556-3471-43c3-9bde-e11fc212301a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea1460c53954c6a915fc638a2690cbf, user_id=1cdee89f8a8040ca944458d3fc6f18eb, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:45.241023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a2556-3471-43c3-9bde-e11fc212301a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:45.380267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1066/2112] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:45 2022] PUT /identity/v3/projects/3ea1460c53954c6a915fc638a2690cbf/users/1cdee89f8a8040ca944458d3fc6f18eb/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:45.386328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fa7d2-a3a0-4352-8ffa-0c60cab05cf5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:45.386621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fa7d2-a3a0-4352-8ffa-0c60cab05cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:45.386905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fa7d2-a3a0-4352-8ffa-0c60cab05cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:45.403917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-3c7fa7d2-a3a0-4352-8ffa-0c60cab05cf5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:45.405284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3c7fa7d2-a3a0-4352-8ffa-0c60cab05cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:45.426429 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c586081b7ee46d9ad9c973b7b28056a', 'request_id': 'req-7aa3b32e-e329-4841-9a54-1149dd40c88c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:45.432614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c586081b7ee46d9ad9c973b7b28056a', 'request_id': 'req-7aa3b32e-e329-4841-9a54-1149dd40c88c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:45.465496 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1067/2113] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:45.474881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fa7d2-a3a0-4352-8ffa-0c60cab05cf5 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:45.477123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-3c7fa7d2-a3a0-4352-8ffa-0c60cab05cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:45.533452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fa7d2-a3a0-4352-8ffa-0c60cab05cf5 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:45.557343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c586081b7ee46d9ad9c973b7b28056a', 'request_id': 'req-7aa3b32e-e329-4841-9a54-1149dd40c88c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:45.571557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] The token cache is being invalidate because project 0c586081b7ee46d9ad9c973b7b28056a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:45.571846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:45.573300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:45.580528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1047/2114] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:45 2022] DELETE /identity/v3/projects/0c586081b7ee46d9ad9c973b7b28056a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:45.581532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:45.583055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:45.584479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:45.585335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:45.585685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:45.586055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:45.591852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:45.652865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3b32e-e329-4841-9a54-1149dd40c88c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:45.657512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:45.658409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:45.658691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:45.658964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] PATH_INFO: `/v3/users/58c9304fbbaa49bd8e0e3db559919747` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:45.660152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c9304fbbaa49bd8e0e3db559919747)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:45.662097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:45.664850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:45.666526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:45.676541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1068/2115] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:45.928680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c9304fbbaa49bd8e0e3db559919747'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:45.929754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c9304fbbaa49bd8e0e3db559919747'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:45.930840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c9304fbbaa49bd8e0e3db559919747'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:45.939210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c9304fbbaa49bd8e0e3db559919747'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:46.033761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c9304fbbaa49bd8e0e3db559919747'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:46.053287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1048/2116] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:45 2022] DELETE /identity/v3/users/58c9304fbbaa49bd8e0e3db559919747 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:46.595986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:46.606349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bd951-1377-4a5c-be69-8f4870ea4221 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:46.671067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:46.671857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:46.672139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:46.672411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] PATH_INFO: `/v3/projects/8971f782c28842b18817e47a48e4b9d7` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:46.673604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8971f782c28842b18817e47a48e4b9d7)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:46.675582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:46.894500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8971f782c28842b18817e47a48e4b9d7', 'request_id': 'req-586c55e4-ff6d-4dc4-9784-13b61a07adc0'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:46.900193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8971f782c28842b18817e47a48e4b9d7', 'request_id': 'req-586c55e4-ff6d-4dc4-9784-13b61a07adc0'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:46.932463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:46.934057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:46.954620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851db74-cb42-400d-afcd-d74db9c69a98 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.016843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.017713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.018019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.018321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] PATH_INFO: `/v3/users/32afde1423bf4983a6070c9203510d7e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.019520 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32afde1423bf4983a6070c9203510d7e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.021432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:47.050677 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8971f782c28842b18817e47a48e4b9d7', 'request_id': 'req-586c55e4-ff6d-4dc4-9784-13b61a07adc0'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:47.062711 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] The token cache is being invalidate because project 8971f782c28842b18817e47a48e4b9d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:47.063001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:47.063640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-586c55e4-ff6d-4dc4-9784-13b61a07adc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:47.065154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1069/2117] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:46 2022] DELETE /identity/v3/projects/8971f782c28842b18817e47a48e4b9d7 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:47.069957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b383-b7a1-491d-8e06-1b2bc321a481 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.070247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b383-b7a1-491d-8e06-1b2bc321a481 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.070523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b383-b7a1-491d-8e06-1b2bc321a481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.089436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f5f1b383-b7a1-491d-8e06-1b2bc321a481 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:47.090704 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f5f1b383-b7a1-491d-8e06-1b2bc321a481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:47.098675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1070/2118] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:47.106894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0d254-f2bd-4fe1-bb55-5341aa950865 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.107183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0d254-f2bd-4fe1-bb55-5341aa950865 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.107466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0d254-f2bd-4fe1-bb55-5341aa950865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.125384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-04d0d254-f2bd-4fe1-bb55-5341aa950865 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:47.126600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-04d0d254-f2bd-4fe1-bb55-5341aa950865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:47.134490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1071/2119] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:47.143481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0d254-f2bd-4fe1-bb55-5341aa950865 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:47.152408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0d254-f2bd-4fe1-bb55-5341aa950865 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.157008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.157669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.158252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.158917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.159378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:47.159691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.161405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:47.172944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1072/2120] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:47.177031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:47.195034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dce90-90f6-4a58-aacd-715719a2dbe4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.203192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10ebeb-5fe6-44df-aa12-f9474095e07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.203192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10ebeb-5fe6-44df-aa12-f9474095e07e admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.203192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10ebeb-5fe6-44df-aa12-f9474095e07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.203192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10ebeb-5fe6-44df-aa12-f9474095e07e admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.204300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10ebeb-5fe6-44df-aa12-f9474095e07e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.206284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10ebeb-5fe6-44df-aa12-f9474095e07e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:47.265199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32afde1423bf4983a6070c9203510d7e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:47.265999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32afde1423bf4983a6070c9203510d7e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:47.266673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32afde1423bf4983a6070c9203510d7e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:47.273486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32afde1423bf4983a6070c9203510d7e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:47.344167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1073/2121] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:47 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:47.346572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70e956-cda5-4523-b5f8-b03f126ca0b1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.346572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70e956-cda5-4523-b5f8-b03f126ca0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.346572 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70e956-cda5-4523-b5f8-b03f126ca0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.382617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-4c70e956-cda5-4523-b5f8-b03f126ca0b1 None None] MFA Rules not processed for user `1cdee89f8a8040ca944458d3fc6f18eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:47.383867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4c70e956-cda5-4523-b5f8-b03f126ca0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:47.397249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32afde1423bf4983a6070c9203510d7e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:47.418268 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1049/2122] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:46 2022] DELETE /identity/v3/users/32afde1423bf4983a6070c9203510d7e => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:47.418268 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:47.418268 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:47.448749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1074/2123] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:47.449999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975ca3d-6d9e-42cf-a621-ded5def4587f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.450296 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975ca3d-6d9e-42cf-a621-ded5def4587f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.450576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975ca3d-6d9e-42cf-a621-ded5def4587f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.467734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6975ca3d-6d9e-42cf-a621-ded5def4587f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:47.468941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6975ca3d-6d9e-42cf-a621-ded5def4587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:47.486069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12d78f-66ba-4d0b-84fc-1bd6e98e4f63 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.490746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.491407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.491967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.491967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.492663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.494640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:47.529228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1075/2124] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:47.530260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975ca3d-6d9e-42cf-a621-ded5def4587f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:47.531526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6975ca3d-6d9e-42cf-a621-ded5def4587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:47.587045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975ca3d-6d9e-42cf-a621-ded5def4587f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.590731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26239e90-ac7f-4dba-ba66-b77b8ecdfb5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.591375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26239e90-ac7f-4dba-ba66-b77b8ecdfb5e service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.591663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26239e90-ac7f-4dba-ba66-b77b8ecdfb5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.591943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26239e90-ac7f-4dba-ba66-b77b8ecdfb5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.647286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26239e90-ac7f-4dba-ba66-b77b8ecdfb5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.648863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26239e90-ac7f-4dba-ba66-b77b8ecdfb5e service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:47.654562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1050/2125] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:47 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:47.655465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:47.657373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:47.657996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1076/2126] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:47.658823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26239e90-ac7f-4dba-ba66-b77b8ecdfb5e service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:47.669432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26239e90-ac7f-4dba-ba66-b77b8ecdfb5e service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.674131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333878d-99c3-4768-b8e5-9c65f27d08f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.674725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333878d-99c3-4768-b8e5-9c65f27d08f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.675004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333878d-99c3-4768-b8e5-9c65f27d08f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.675276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333878d-99c3-4768-b8e5-9c65f27d08f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.676186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333878d-99c3-4768-b8e5-9c65f27d08f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.677811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333878d-99c3-4768-b8e5-9c65f27d08f5 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:47.688486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1077/2127] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:47.692133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333878d-99c3-4768-b8e5-9c65f27d08f5 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:47.703948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333878d-99c3-4768-b8e5-9c65f27d08f5 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.710115 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bba63-50ff-450f-b76a-1d9bbd15beac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.711528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bba63-50ff-450f-b76a-1d9bbd15beac admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.711828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bba63-50ff-450f-b76a-1d9bbd15beac admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.712155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bba63-50ff-450f-b76a-1d9bbd15beac admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.713004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bba63-50ff-450f-b76a-1d9bbd15beac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.714830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bba63-50ff-450f-b76a-1d9bbd15beac admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:47.720106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ff9aa-b070-4c03-a21f-d794cf0047d6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.724031 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a47c3-1f17-4bcc-a15a-64d6c31e50a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.724707 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a47c3-1f17-4bcc-a15a-64d6c31e50a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.724991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a47c3-1f17-4bcc-a15a-64d6c31e50a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.725272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a47c3-1f17-4bcc-a15a-64d6c31e50a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.729406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1078/2128] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:47.737238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a47c3-1f17-4bcc-a15a-64d6c31e50a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.740132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bba63-50ff-450f-b76a-1d9bbd15beac admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:47.740708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a47c3-1f17-4bcc-a15a-64d6c31e50a2 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:47.750939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bba63-50ff-450f-b76a-1d9bbd15beac admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:47.753921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1051/2129] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:47.756723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46821581-89a2-428c-9294-bff65ad0e208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:47.757816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821581-89a2-428c-9294-bff65ad0e208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:47.758116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821581-89a2-428c-9294-bff65ad0e208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:47.758396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821581-89a2-428c-9294-bff65ad0e208 admin admin] PATH_INFO: `/v3/projects/6b9ac2a707db4af9a53e4f660a0af21b/users/68dee9e64bd7422ba2a4d9b904e18df2/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:47.774481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46821581-89a2-428c-9294-bff65ad0e208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b9ac2a707db4af9a53e4f660a0af21b, user_id=68dee9e64bd7422ba2a4d9b904e18df2, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:47.777274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46821581-89a2-428c-9294-bff65ad0e208 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.014703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1079/2130] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:47 2022] PUT /identity/v3/projects/6b9ac2a707db4af9a53e4f660a0af21b/users/68dee9e64bd7422ba2a4d9b904e18df2/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:48.026692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05330c69-0850-457a-9859-77dba7a6bdb3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.027109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05330c69-0850-457a-9859-77dba7a6bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.027254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05330c69-0850-457a-9859-77dba7a6bdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.056458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-05330c69-0850-457a-9859-77dba7a6bdb3 None None] MFA Rules not processed for user `68dee9e64bd7422ba2a4d9b904e18df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:48.057704 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-05330c69-0850-457a-9859-77dba7a6bdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:48.133535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1052/2131] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:48.138748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46821581-89a2-428c-9294-bff65ad0e208 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.202879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46821581-89a2-428c-9294-bff65ad0e208 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.208118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360c3b-c0d3-45fb-8a8c-2c8bd41701d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.208791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360c3b-c0d3-45fb-8a8c-2c8bd41701d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.209078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360c3b-c0d3-45fb-8a8c-2c8bd41701d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.209350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360c3b-c0d3-45fb-8a8c-2c8bd41701d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.210085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39360c3b-c0d3-45fb-8a8c-2c8bd41701d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.211993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39360c3b-c0d3-45fb-8a8c-2c8bd41701d1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.325637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05330c69-0850-457a-9859-77dba7a6bdb3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.327573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-05330c69-0850-457a-9859-77dba7a6bdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:48.330985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1080/2132] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:48 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:48.331735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360c3b-c0d3-45fb-8a8c-2c8bd41701d1 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.364268 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05330c69-0850-457a-9859-77dba7a6bdb3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.408562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360c3b-c0d3-45fb-8a8c-2c8bd41701d1 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.413511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760109-89fe-4766-9ef4-9a5a7f2c275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.414243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760109-89fe-4766-9ef4-9a5a7f2c275f admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.414537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760109-89fe-4766-9ef4-9a5a7f2c275f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.414817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760109-89fe-4766-9ef4-9a5a7f2c275f admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.415511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16760109-89fe-4766-9ef4-9a5a7f2c275f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.417587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16760109-89fe-4766-9ef4-9a5a7f2c275f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.425376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.426361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.427500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.427500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] PATH_INFO: `/v3/projects/8a768d97ae0944f19d69318e32c9948d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.428672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a768d97ae0944f19d69318e32c9948d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.430601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.517441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1081/2133] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:48 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:48.521865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760109-89fe-4766-9ef4-9a5a7f2c275f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.529900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a768d97ae0944f19d69318e32c9948d', 'request_id': 'req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:48.533251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760109-89fe-4766-9ef4-9a5a7f2c275f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.535402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a768d97ae0944f19d69318e32c9948d', 'request_id': 'req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:48.593243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc1365-d95c-4732-9fbe-34299a85e016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.593844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc1365-d95c-4732-9fbe-34299a85e016 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.594159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc1365-d95c-4732-9fbe-34299a85e016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.594408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc1365-d95c-4732-9fbe-34299a85e016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.595127 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc1365-d95c-4732-9fbe-34299a85e016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.596720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc1365-d95c-4732-9fbe-34299a85e016 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.607184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1082/2134] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:48.611094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc1365-d95c-4732-9fbe-34299a85e016 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.620834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc1365-d95c-4732-9fbe-34299a85e016 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.625606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.626502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.626793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.627070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] PATH_INFO: `/v3/projects/5df515c3ca9b447d95509295acbee56c/users/8b29fbac1b5142429360db503f2866fa/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.639196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a768d97ae0944f19d69318e32c9948d', 'request_id': 'req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:48.641891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df515c3ca9b447d95509295acbee56c, user_id=8b29fbac1b5142429360db503f2866fa, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.644570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.651064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] The token cache is being invalidate because project 8a768d97ae0944f19d69318e32c9948d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:48.651417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:48.652212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-88b0a162-fc61-43e0-81a4-3a74f6eb6d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:48.654455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1053/2135] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:48 2022] DELETE /identity/v3/projects/8a768d97ae0944f19d69318e32c9948d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:48.659173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdddf93-ece6-440d-8d08-a32cd74b6c09 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.659463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdddf93-ece6-440d-8d08-a32cd74b6c09 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.659788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdddf93-ece6-440d-8d08-a32cd74b6c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.678737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3cdddf93-ece6-440d-8d08-a32cd74b6c09 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:48.679946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3cdddf93-ece6-440d-8d08-a32cd74b6c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:48.687756 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1054/2136] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:48.696495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0582072-67e2-44d8-995c-441b222adfa4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.696791 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0582072-67e2-44d8-995c-441b222adfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.697076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0582072-67e2-44d8-995c-441b222adfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.715152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b0582072-67e2-44d8-995c-441b222adfa4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:48.716878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b0582072-67e2-44d8-995c-441b222adfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:48.724984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1083/2137] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:48 2022] PUT /identity/v3/projects/5df515c3ca9b447d95509295acbee56c/users/8b29fbac1b5142429360db503f2866fa/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:48.729043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.730842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:48.779287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1055/2138] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:48.783480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0582072-67e2-44d8-995c-441b222adfa4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.792418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0582072-67e2-44d8-995c-441b222adfa4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.793463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c979b7-25fc-4728-90c8-945abc54bae1 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.797660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.798337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.798615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.798881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43cd3-0a26-4bc9-86c2-cca49859f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.799010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.799549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43cd3-0a26-4bc9-86c2-cca49859f38e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.799658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:48.799832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43cd3-0a26-4bc9-86c2-cca49859f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.799930 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.800108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43cd3-0a26-4bc9-86c2-cca49859f38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.800830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe43cd3-0a26-4bc9-86c2-cca49859f38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.801491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.802580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe43cd3-0a26-4bc9-86c2-cca49859f38e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.812920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1084/2139] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:48.816506 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1056/2140] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:48.816801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43cd3-0a26-4bc9-86c2-cca49859f38e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.817221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.827475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43cd3-0a26-4bc9-86c2-cca49859f38e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.827679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f98e84-01fb-446c-952d-5d7a1954c796 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:48.832168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80db0e-9dc2-46c6-bddc-db391b62140f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.832301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b90902-fd5f-4a25-9da4-144fca340607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:48.832807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80db0e-9dc2-46c6-bddc-db391b62140f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.833031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b90902-fd5f-4a25-9da4-144fca340607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.833136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80db0e-9dc2-46c6-bddc-db391b62140f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.833312 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b90902-fd5f-4a25-9da4-144fca340607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.833404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80db0e-9dc2-46c6-bddc-db391b62140f admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.833583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b90902-fd5f-4a25-9da4-144fca340607 admin admin] PATH_INFO: `/v3/projects/5df515c3ca9b447d95509295acbee56c/users/8b29fbac1b5142429360db503f2866fa/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.834246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80db0e-9dc2-46c6-bddc-db391b62140f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.835609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b90902-fd5f-4a25-9da4-144fca340607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df515c3ca9b447d95509295acbee56c, user_id=8b29fbac1b5142429360db503f2866fa, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:48.836155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80db0e-9dc2-46c6-bddc-db391b62140f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.838369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b90902-fd5f-4a25-9da4-144fca340607 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:48.849098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1085/2141] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:48 2022] PUT /identity/v3/projects/5df515c3ca9b447d95509295acbee56c/users/8b29fbac1b5142429360db503f2866fa/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:48.859103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189060-c523-498b-bed6-e12f5903dc22 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:48.859464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189060-c523-498b-bed6-e12f5903dc22 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:48.859673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189060-c523-498b-bed6-e12f5903dc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:48.884915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ba189060-c523-498b-bed6-e12f5903dc22 None None] MFA Rules not processed for user `8b29fbac1b5142429360db503f2866fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:48.886153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ba189060-c523-498b-bed6-e12f5903dc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:48.951741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1086/2142] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:48.967785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1057/2143] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:48 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:48.970996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189060-c523-498b-bed6-e12f5903dc22 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:48.970996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ba189060-c523-498b-bed6-e12f5903dc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:48.971787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80db0e-9dc2-46c6-bddc-db391b62140f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.027234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189060-c523-498b-bed6-e12f5903dc22 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.033903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80db0e-9dc2-46c6-bddc-db391b62140f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.038683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca47c-3db3-406e-9ed3-a6ea9bd3bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.039345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca47c-3db3-406e-9ed3-a6ea9bd3bef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.039631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca47c-3db3-406e-9ed3-a6ea9bd3bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.039912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eca47c-3db3-406e-9ed3-a6ea9bd3bef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.040599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eca47c-3db3-406e-9ed3-a6ea9bd3bef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.042561 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eca47c-3db3-406e-9ed3-a6ea9bd3bef4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.077128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672d78b-6d2a-4884-8165-b3ce579270bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.077778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672d78b-6d2a-4884-8165-b3ce579270bd service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.078116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672d78b-6d2a-4884-8165-b3ce579270bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.078392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672d78b-6d2a-4884-8165-b3ce579270bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.135455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672d78b-6d2a-4884-8165-b3ce579270bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.136906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672d78b-6d2a-4884-8165-b3ce579270bd service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.142880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1058/2144] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:48 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:49.146685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca47c-3db3-406e-9ed3-a6ea9bd3bef4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.150432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1087/2145] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:49.157307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eca47c-3db3-406e-9ed3-a6ea9bd3bef4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.162030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a519d-d363-4719-a604-1dcd913012ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.162692 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a519d-d363-4719-a604-1dcd913012ac admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.162981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a519d-d363-4719-a604-1dcd913012ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.163395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a519d-d363-4719-a604-1dcd913012ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.164136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a519d-d363-4719-a604-1dcd913012ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.165767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a519d-d363-4719-a604-1dcd913012ac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.176176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1059/2146] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:49.180141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672d78b-6d2a-4884-8165-b3ce579270bd service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.191378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672d78b-6d2a-4884-8165-b3ce579270bd service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.196475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4099b-90ee-442c-86fb-0d0d5dd4695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.197218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4099b-90ee-442c-86fb-0d0d5dd4695a admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.197884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4099b-90ee-442c-86fb-0d0d5dd4695a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.198237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4099b-90ee-442c-86fb-0d0d5dd4695a admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.199087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4099b-90ee-442c-86fb-0d0d5dd4695a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.201338 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4099b-90ee-442c-86fb-0d0d5dd4695a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.214614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1088/2147] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:49.215832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a519d-d363-4719-a604-1dcd913012ac admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.226169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a519d-d363-4719-a604-1dcd913012ac admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.230909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ef76b-f788-4b67-9dd8-71cbb26c7285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.232186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ef76b-f788-4b67-9dd8-71cbb26c7285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.232539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ef76b-f788-4b67-9dd8-71cbb26c7285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.232888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ef76b-f788-4b67-9dd8-71cbb26c7285 admin admin] PATH_INFO: `/v3/projects/4114a6458a10471aa7d5518a76921721/users/0546ab5ad5904f009f5f3814fbe9874f/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.249306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ef76b-f788-4b67-9dd8-71cbb26c7285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4114a6458a10471aa7d5518a76921721, user_id=0546ab5ad5904f009f5f3814fbe9874f, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.252077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ef76b-f788-4b67-9dd8-71cbb26c7285 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.363835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1060/2148] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:49 2022] PUT /identity/v3/projects/4114a6458a10471aa7d5518a76921721/users/0546ab5ad5904f009f5f3814fbe9874f/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:49.374808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8381712-3088-4e40-a8bd-ef00fc935959 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.375118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8381712-3088-4e40-a8bd-ef00fc935959 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.375439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8381712-3088-4e40-a8bd-ef00fc935959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.407039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d8381712-3088-4e40-a8bd-ef00fc935959 None None] MFA Rules not processed for user `0546ab5ad5904f009f5f3814fbe9874f`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:49.408342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d8381712-3088-4e40-a8bd-ef00fc935959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:49.473866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1089/2149] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:49.480231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ef76b-f788-4b67-9dd8-71cbb26c7285 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.547702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ef76b-f788-4b67-9dd8-71cbb26c7285 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.552789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdf7e0-59e1-4152-9605-ca245274d95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.553164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdf7e0-59e1-4152-9605-ca245274d95a admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.553450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdf7e0-59e1-4152-9605-ca245274d95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.553729 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdf7e0-59e1-4152-9605-ca245274d95a admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.554461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdf7e0-59e1-4152-9605-ca245274d95a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.556523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdf7e0-59e1-4152-9605-ca245274d95a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.561425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8381712-3088-4e40-a8bd-ef00fc935959 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.562851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d8381712-3088-4e40-a8bd-ef00fc935959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:49.577580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8381712-3088-4e40-a8bd-ef00fc935959 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.582530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.583404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.583690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.583964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] PATH_INFO: `/v3/users/1bbf4e1b0bc14b038c2ac580d61f673f` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.585165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bbf4e1b0bc14b038c2ac580d61f673f)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.587277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.678731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1061/2150] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:49 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:49.680017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40766-48d2-4b04-b5df-d93248bf96f7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.684996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40766-48d2-4b04-b5df-d93248bf96f7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.684996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40766-48d2-4b04-b5df-d93248bf96f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.698224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3fb40766-48d2-4b04-b5df-d93248bf96f7 None None] MFA Rules not processed for user `1cdee89f8a8040ca944458d3fc6f18eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:49.699431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3fb40766-48d2-4b04-b5df-d93248bf96f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:49.753835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1062/2151] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:49.754706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb40766-48d2-4b04-b5df-d93248bf96f7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.818233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb40766-48d2-4b04-b5df-d93248bf96f7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.823282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.828074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.828074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.828074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.828074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.828074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.837597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bbf4e1b0bc14b038c2ac580d61f673f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:49.838472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bbf4e1b0bc14b038c2ac580d61f673f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:49.936765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bbf4e1b0bc14b038c2ac580d61f673f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:49.943482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1063/2152] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:49 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:49.945164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bbf4e1b0bc14b038c2ac580d61f673f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:49.945882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de72b331-db8a-450a-a0d2-7b1eafa5882c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bbf4e1b0bc14b038c2ac580d61f673f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:49.947772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.951826 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:49.962615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2a367-7b7f-49b4-8a05-55cfc7da85ae admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:49.964901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1090/2153] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:49 2022] DELETE /identity/v3/users/1bbf4e1b0bc14b038c2ac580d61f673f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:49.968188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32801ef-ab58-4c33-a40a-5a8eaa287dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:49.968862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32801ef-ab58-4c33-a40a-5a8eaa287dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.969145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32801ef-ab58-4c33-a40a-5a8eaa287dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.969580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32801ef-ab58-4c33-a40a-5a8eaa287dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.970470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32801ef-ab58-4c33-a40a-5a8eaa287dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:49.971685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d88b06-aeb9-474f-a3e7-6299872f3672 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:49.971987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d88b06-aeb9-474f-a3e7-6299872f3672 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:49.972106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32801ef-ab58-4c33-a40a-5a8eaa287dfd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:49.972325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d88b06-aeb9-474f-a3e7-6299872f3672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:49.986116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1064/2154] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:49.987838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32801ef-ab58-4c33-a40a-5a8eaa287dfd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:49.990566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-78d88b06-aeb9-474f-a3e7-6299872f3672 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:49.991783 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-78d88b06-aeb9-474f-a3e7-6299872f3672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:49.998323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32801ef-ab58-4c33-a40a-5a8eaa287dfd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.056230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1091/2155] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:50.062079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aee17-f1ce-408b-b781-ad4afddc8755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.063116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aee17-f1ce-408b-b781-ad4afddc8755 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.063403 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aee17-f1ce-408b-b781-ad4afddc8755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.064238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aee17-f1ce-408b-b781-ad4afddc8755 admin admin] PATH_INFO: `/v3/projects/796cb64587064358b1d24d50e389ee14/users/86ca0c8f88114811ad49bffe16c25714/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.066852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d88b06-aeb9-474f-a3e7-6299872f3672 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:50.068222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-78d88b06-aeb9-474f-a3e7-6299872f3672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:50.081830 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05aee17-f1ce-408b-b781-ad4afddc8755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796cb64587064358b1d24d50e389ee14, user_id=86ca0c8f88114811ad49bffe16c25714, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.084544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05aee17-f1ce-408b-b781-ad4afddc8755 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.125421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d88b06-aeb9-474f-a3e7-6299872f3672 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.129261 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26d25d-db75-409b-9eae-5458e276d01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.129894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26d25d-db75-409b-9eae-5458e276d01c service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.130191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26d25d-db75-409b-9eae-5458e276d01c service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.130447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26d25d-db75-409b-9eae-5458e276d01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.139380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26d25d-db75-409b-9eae-5458e276d01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.140899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26d25d-db75-409b-9eae-5458e276d01c service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.151576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1092/2156] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:50.176186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1065/2157] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:49 2022] PUT /identity/v3/projects/796cb64587064358b1d24d50e389ee14/users/86ca0c8f88114811ad49bffe16c25714/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:50.176661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26d25d-db75-409b-9eae-5458e276d01c service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:50.187688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26d25d-db75-409b-9eae-5458e276d01c service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.249282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0885f7-d0a6-4e2b-bdf5-f619739a44dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.250051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0885f7-d0a6-4e2b-bdf5-f619739a44dd admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.250339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0885f7-d0a6-4e2b-bdf5-f619739a44dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.250702 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0885f7-d0a6-4e2b-bdf5-f619739a44dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.251465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0885f7-d0a6-4e2b-bdf5-f619739a44dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.253159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0885f7-d0a6-4e2b-bdf5-f619739a44dd admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.263070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1093/2158] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:50.267309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aee17-f1ce-408b-b781-ad4afddc8755 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:50.277526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aee17-f1ce-408b-b781-ad4afddc8755 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.282929 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.285238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.285238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.285238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] PATH_INFO: `/v3/projects/796cb64587064358b1d24d50e389ee14/users/86ca0c8f88114811ad49bffe16c25714/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.286473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796cb64587064358b1d24d50e389ee14, user_id=86ca0c8f88114811ad49bffe16c25714, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.289239 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.299980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1066/2159] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:50 2022] PUT /identity/v3/projects/796cb64587064358b1d24d50e389ee14/users/86ca0c8f88114811ad49bffe16c25714/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:50.310167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73518ae8-972b-424d-a8bf-e08049d6dbff None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.310464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73518ae8-972b-424d-a8bf-e08049d6dbff None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.313948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73518ae8-972b-424d-a8bf-e08049d6dbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.341276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-73518ae8-972b-424d-a8bf-e08049d6dbff None None] MFA Rules not processed for user `86ca0c8f88114811ad49bffe16c25714`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:50.342583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-73518ae8-972b-424d-a8bf-e08049d6dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:50.404789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:50.406916 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:50.459431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1094/2160] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:50.471715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73518ae8-972b-424d-a8bf-e08049d6dbff None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:50.473426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-73518ae8-972b-424d-a8bf-e08049d6dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:50.505444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690e9ed-9070-41f3-a119-1358e933c6b4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.512842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b29fbac1b5142429360db503f2866fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5df515c3ca9b447d95509295acbee56c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.513517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.513905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.514198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.514945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:50.515259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.516886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.533330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1067/2161] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:50.536968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:50.546228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73518ae8-972b-424d-a8bf-e08049d6dbff None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.553687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbd1fb-042f-48ff-9a70-1c0aa8bfe25b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.553687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbd1fb-042f-48ff-9a70-1c0aa8bfe25b service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.553687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbd1fb-042f-48ff-9a70-1c0aa8bfe25b service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.553687 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbd1fb-042f-48ff-9a70-1c0aa8bfe25b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.554295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280d499-109e-4457-b1b6-bb2321261a05 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.563626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbd1fb-042f-48ff-9a70-1c0aa8bfe25b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.565993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbd1fb-042f-48ff-9a70-1c0aa8bfe25b service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.566896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b29fbac1b5142429360db503f2866fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5df515c3ca9b447d95509295acbee56c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.567673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.567852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.568267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.568976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.570997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.583213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1095/2162] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:50.665461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbd1fb-042f-48ff-9a70-1c0aa8bfe25b service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:50.728546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbd1fb-042f-48ff-9a70-1c0aa8bfe25b service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.735177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.736063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.736348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.736624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] PATH_INFO: `/v3/users/0546ab5ad5904f009f5f3814fbe9874f` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.737894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0546ab5ad5904f009f5f3814fbe9874f)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.739818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.766288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1068/2163] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:50 2022] POST /identity/v3/projects => generated 353 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:50.772719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:50.772719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:50.846543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cae66-3357-4775-a022-136355f0368e tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:50.851251 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:50.852118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:50.852404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:50.852683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] PATH_INFO: `/v3/projects/ae10ab699bed431b808aafa89d940578` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:50.853936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae10ab699bed431b808aafa89d940578)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:50.855867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:50.953923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0546ab5ad5904f009f5f3814fbe9874f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:50.954592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0546ab5ad5904f009f5f3814fbe9874f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:50.981443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae10ab699bed431b808aafa89d940578', 'request_id': 'req-fe0c7d34-40ef-4d46-911c-79be28bc88bd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:50.986681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae10ab699bed431b808aafa89d940578', 'request_id': 'req-fe0c7d34-40ef-4d46-911c-79be28bc88bd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.022493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0546ab5ad5904f009f5f3814fbe9874f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.030020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0546ab5ad5904f009f5f3814fbe9874f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.030672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0546ab5ad5904f009f5f3814fbe9874f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.047455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1096/2164] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:50 2022] DELETE /identity/v3/users/0546ab5ad5904f009f5f3814fbe9874f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:51.050595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:51.050595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:51.079472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae10ab699bed431b808aafa89d940578', 'request_id': 'req-fe0c7d34-40ef-4d46-911c-79be28bc88bd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.091573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] The token cache is being invalidate because project ae10ab699bed431b808aafa89d940578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:51.091859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.092731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0c7d34-40ef-4d46-911c-79be28bc88bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.094270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1069/2165] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:50 2022] DELETE /identity/v3/projects/ae10ab699bed431b808aafa89d940578 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:51.095492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6374-71d8-42bc-b67e-883a4f26cc5c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:51.095798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6374-71d8-42bc-b67e-883a4f26cc5c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:51.096052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6374-71d8-42bc-b67e-883a4f26cc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:51.113687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3bde6374-71d8-42bc-b67e-883a4f26cc5c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:51.114900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3bde6374-71d8-42bc-b67e-883a4f26cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:51.118839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101a5c7-0135-44a7-995e-239bf234be04 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:51.124486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b29fbac1b5142429360db503f2866fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5df515c3ca9b447d95509295acbee56c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:51.125491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:51.125920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:51.126333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:51.127406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:51.127918 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:51.129830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:51.142087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1097/2166] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:26:51.142997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:51.144719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:51.190203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1070/2167] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:51.192606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde6374-71d8-42bc-b67e-883a4f26cc5c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:51.194365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3bde6374-71d8-42bc-b67e-883a4f26cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:51.209337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde6374-71d8-42bc-b67e-883a4f26cc5c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:51.214059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b29fbac1b5142429360db503f2866fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5df515c3ca9b447d95509295acbee56c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:51.214977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:51.215266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:51.215603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] PATH_INFO: `/v3/projects/2450cc94f9f14f8ea2564bca3c08e472` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:51.216807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2450cc94f9f14f8ea2564bca3c08e472)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:51.218774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:51.221447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9e0b9-2bf1-47f9-81c5-060620b7d4b3 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:51.226028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:51.226887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:51.227165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:51.227434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] PATH_INFO: `/v3/users/cc236d63727d40368dd663a3544324f9` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:51.228625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc236d63727d40368dd663a3544324f9)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:51.230632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:51.393810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2450cc94f9f14f8ea2564bca3c08e472', 'request_id': 'req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.399207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2450cc94f9f14f8ea2564bca3c08e472', 'request_id': 'req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.479694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2450cc94f9f14f8ea2564bca3c08e472', 'request_id': 'req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.491932 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] The token cache is being invalidate because project 2450cc94f9f14f8ea2564bca3c08e472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:51.492229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.493237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.494621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1071/2168] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:51 2022] DELETE /identity/v3/projects/2450cc94f9f14f8ea2564bca3c08e472 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:51.495490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:51.496460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc236d63727d40368dd663a3544324f9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.498172 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:51.499036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc236d63727d40368dd663a3544324f9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.557100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac4b83-4b3f-488c-bc8f-998e0cb7e829 tempest-VolumeQuotaClassesTest-502163902 tempest-VolumeQuotaClassesTest-502163902-project-admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:51.561322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:51.562009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:51.562296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:51.562571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:51.564708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:51.627924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:51.629367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:51.631535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc236d63727d40368dd663a3544324f9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.639483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1072/2169] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:51.639800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc236d63727d40368dd663a3544324f9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.640605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc236d63727d40368dd663a3544324f9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:51.657616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1098/2170] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:51 2022] DELETE /identity/v3/users/cc236d63727d40368dd663a3544324f9 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:52.207867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:52.209619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:52.303413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cef64-49dd-4b1b-9aaf-4f1782439eba service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:52.358866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:52.360471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:52.360797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:52.365653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:52.365653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:52.365653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] PATH_INFO: `/v3/projects/4114a6458a10471aa7d5518a76921721` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:52.365653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4114a6458a10471aa7d5518a76921721)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:52.365653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:52.408758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b049-a5ca-4e2b-a65e-4c0c0431a4c7 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:52.413427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:52.414422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:52.414706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:52.414978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] PATH_INFO: `/v3/users/68dee9e64bd7422ba2a4d9b904e18df2` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:52.416207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68dee9e64bd7422ba2a4d9b904e18df2)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:52.418167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:52.502237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4114a6458a10471aa7d5518a76921721', 'request_id': 'req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.507471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4114a6458a10471aa7d5518a76921721', 'request_id': 'req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.654551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4114a6458a10471aa7d5518a76921721', 'request_id': 'req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.666277 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] The token cache is being invalidate because project 4114a6458a10471aa7d5518a76921721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:52.666573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.667226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.668778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1073/2171] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:52 2022] DELETE /identity/v3/projects/4114a6458a10471aa7d5518a76921721 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:52.671772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:52.671772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:52.732118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ece32-3bb0-4b0e-bf7f-ee36e16c4515 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:52.736934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:52.737821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:52.738108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:52.738387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] PATH_INFO: `/v3/projects/0b04f57c5c3e4702b0d175d72596261c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:52.739579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b04f57c5c3e4702b0d175d72596261c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:52.741491 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:52.794731 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68dee9e64bd7422ba2a4d9b904e18df2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.795393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68dee9e64bd7422ba2a4d9b904e18df2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.885142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b04f57c5c3e4702b0d175d72596261c', 'request_id': 'req-5d599917-8d47-4fc6-af69-351b6918a761'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.891265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b04f57c5c3e4702b0d175d72596261c', 'request_id': 'req-5d599917-8d47-4fc6-af69-351b6918a761'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.919153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68dee9e64bd7422ba2a4d9b904e18df2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.926395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68dee9e64bd7422ba2a4d9b904e18df2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.927020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68dee9e64bd7422ba2a4d9b904e18df2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:52.943714 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1099/2172] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:52 2022] DELETE /identity/v3/users/68dee9e64bd7422ba2a4d9b904e18df2 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:52.946785 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:52.946785 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:53.008553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c431a-63a4-46c8-a154-64e6c82e6e61 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.013972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.014830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.015133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.015406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] PATH_INFO: `/v3/users/86ca0c8f88114811ad49bffe16c25714` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.016614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ca0c8f88114811ad49bffe16c25714)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.018563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:53.033761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b04f57c5c3e4702b0d175d72596261c', 'request_id': 'req-5d599917-8d47-4fc6-af69-351b6918a761'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:53.045755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] The token cache is being invalidate because project 0b04f57c5c3e4702b0d175d72596261c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:53.046075 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:53.046732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5d599917-8d47-4fc6-af69-351b6918a761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:53.049937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1074/2173] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:52 2022] DELETE /identity/v3/projects/0b04f57c5c3e4702b0d175d72596261c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:53.049937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fdd4b-23e0-41e7-b5b3-b4fb4e617a2a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.049937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fdd4b-23e0-41e7-b5b3-b4fb4e617a2a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.050175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fdd4b-23e0-41e7-b5b3-b4fb4e617a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.081249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-130fdd4b-23e0-41e7-b5b3-b4fb4e617a2a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:53.082526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-130fdd4b-23e0-41e7-b5b3-b4fb4e617a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:53.091388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1075/2174] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:53.092713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ada55-8619-4eb1-a94f-a0da762dc33a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.093003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ada55-8619-4eb1-a94f-a0da762dc33a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.093279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ada55-8619-4eb1-a94f-a0da762dc33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.133517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-263ada55-8619-4eb1-a94f-a0da762dc33a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:53.134763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-263ada55-8619-4eb1-a94f-a0da762dc33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:53.143833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1076/2175] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:53.144684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ada55-8619-4eb1-a94f-a0da762dc33a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:53.146168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-263ada55-8619-4eb1-a94f-a0da762dc33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:53.241389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ca0c8f88114811ad49bffe16c25714'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:53.242130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ca0c8f88114811ad49bffe16c25714'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:53.243995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ada55-8619-4eb1-a94f-a0da762dc33a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.248227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687010a6-572a-477f-ba53-ee5a63e053ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.248871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687010a6-572a-477f-ba53-ee5a63e053ca service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.249157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687010a6-572a-477f-ba53-ee5a63e053ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.249436 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687010a6-572a-477f-ba53-ee5a63e053ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.251323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-687010a6-572a-477f-ba53-ee5a63e053ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:53.264182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687010a6-572a-477f-ba53-ee5a63e053ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.265841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687010a6-572a-477f-ba53-ee5a63e053ca service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:53.275368 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1077/2176] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:53.276696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e9d1-238c-49fd-9e20-f039837537b6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.276982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e9d1-238c-49fd-9e20-f039837537b6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.277263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e9d1-238c-49fd-9e20-f039837537b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.295843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4927e9d1-238c-49fd-9e20-f039837537b6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:53.297094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4927e9d1-238c-49fd-9e20-f039837537b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:53.309709 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1078/2177] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:53.311279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927e9d1-238c-49fd-9e20-f039837537b6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:53.321617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927e9d1-238c-49fd-9e20-f039837537b6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.325899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ca0c8f88114811ad49bffe16c25714'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:53.328992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.329877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.332825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.332825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.332825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:53.332825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.334666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:53.336003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ca0c8f88114811ad49bffe16c25714'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:53.337350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ca0c8f88114811ad49bffe16c25714'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:53.354136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1079/2178] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:26:53.356752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:53.368529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1100/2179] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:52 2022] DELETE /identity/v3/users/86ca0c8f88114811ad49bffe16c25714 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:53.375425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c6ab-6350-4f53-a347-243fbc56e25a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.440861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f69bf-3df6-42f9-bd61-e3e4e48b5e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.441861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f69bf-3df6-42f9-bd61-e3e4e48b5e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.442155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f69bf-3df6-42f9-bd61-e3e4e48b5e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.442978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f69bf-3df6-42f9-bd61-e3e4e48b5e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.443757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f69bf-3df6-42f9-bd61-e3e4e48b5e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.446440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f69bf-3df6-42f9-bd61-e3e4e48b5e92 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:53.573082 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1080/2180] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:53 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:53.577333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:53.590126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b98ca3-61cd-430a-ad49-64636e0cd323 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.645621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec22330-4284-413b-a256-882f02e0b748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.646447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec22330-4284-413b-a256-882f02e0b748 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.646845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec22330-4284-413b-a256-882f02e0b748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.647083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec22330-4284-413b-a256-882f02e0b748 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.647787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec22330-4284-413b-a256-882f02e0b748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.649863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec22330-4284-413b-a256-882f02e0b748 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:53.772831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1101/2181] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:53 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:53.777143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f69bf-3df6-42f9-bd61-e3e4e48b5e92 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:53.788934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f69bf-3df6-42f9-bd61-e3e4e48b5e92 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.794804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059098d2-d36b-4425-976f-44647aafaaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.795618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059098d2-d36b-4425-976f-44647aafaaac admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.795958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059098d2-d36b-4425-976f-44647aafaaac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.796307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059098d2-d36b-4425-976f-44647aafaaac admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.797244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059098d2-d36b-4425-976f-44647aafaaac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.803601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059098d2-d36b-4425-976f-44647aafaaac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:53.819774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec22330-4284-413b-a256-882f02e0b748 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:53.820963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1081/2182] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:53.831235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec22330-4284-413b-a256-882f02e0b748 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.836549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.837344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.837701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.838114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.838933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.841247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:53.854520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1102/2183] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:53.863065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059098d2-d36b-4425-976f-44647aafaaac admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:53.873886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059098d2-d36b-4425-976f-44647aafaaac admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.878629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.879505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.879789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.880067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] PATH_INFO: `/v3/projects/a01bc388c3fb4582a5fd6107058310c0/users/5e83e2f7adfb4f77aabfb3ddae66a3ab/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.895709 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a01bc388c3fb4582a5fd6107058310c0, user_id=5e83e2f7adfb4f77aabfb3ddae66a3ab, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.903595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:53.906573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:53.908411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:53.923011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d50065-3405-4ca7-ae26-aa3220610245 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:53.931474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:53.932428 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:53.932723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:53.933004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] PATH_INFO: `/v3/projects/6b9ac2a707db4af9a53e4f660a0af21b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:53.934693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b9ac2a707db4af9a53e4f660a0af21b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:53.937679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:54.039920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1082/2184] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:53 2022] PUT /identity/v3/projects/a01bc388c3fb4582a5fd6107058310c0/users/5e83e2f7adfb4f77aabfb3ddae66a3ab/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:54.040706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:54.042538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:54.109535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09044f-7bd0-42a0-b6c6-b29084c2339e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:54.117769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:54.117769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.117769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.117769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] PATH_INFO: `/v3/projects/796cb64587064358b1d24d50e389ee14` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.118173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] RBAC: Authorizing `identity:delete_project(project_id=796cb64587064358b1d24d50e389ee14)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:54.119125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:54.158180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b9ac2a707db4af9a53e4f660a0af21b', 'request_id': 'req-a5613fdd-9240-41a6-829d-55b92bce935e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.164813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b9ac2a707db4af9a53e4f660a0af21b', 'request_id': 'req-a5613fdd-9240-41a6-829d-55b92bce935e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.237779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796cb64587064358b1d24d50e389ee14', 'request_id': 'req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.246733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796cb64587064358b1d24d50e389ee14', 'request_id': 'req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.256659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b9ac2a707db4af9a53e4f660a0af21b', 'request_id': 'req-a5613fdd-9240-41a6-829d-55b92bce935e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.271553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] The token cache is being invalidate because project 6b9ac2a707db4af9a53e4f660a0af21b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:54.271932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.272837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5613fdd-9240-41a6-829d-55b92bce935e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.274685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1103/2185] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:53 2022] DELETE /identity/v3/projects/6b9ac2a707db4af9a53e4f660a0af21b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:54.275708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40a1f-a8fb-4058-bde8-becc1facc6eb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.276017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40a1f-a8fb-4058-bde8-becc1facc6eb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.276282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe40a1f-a8fb-4058-bde8-becc1facc6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.303136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-cbe40a1f-a8fb-4058-bde8-becc1facc6eb None None] MFA Rules not processed for user `5e83e2f7adfb4f77aabfb3ddae66a3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:54.304899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-cbe40a1f-a8fb-4058-bde8-becc1facc6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:54.359365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796cb64587064358b1d24d50e389ee14', 'request_id': 'req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.368017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1104/2186] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:54.368853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe40a1f-a8fb-4058-bde8-becc1facc6eb None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:54.375296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] The token cache is being invalidate because project 796cb64587064358b1d24d50e389ee14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:54.375587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.375945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-cbe40a1f-a8fb-4058-bde8-becc1facc6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:54.376243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b9f2dea5-ae08-4b8c-b1ef-c8265609783c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.378034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1083/2187] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:54 2022] DELETE /identity/v3/projects/796cb64587064358b1d24d50e389ee14 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:54.382542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85319-dc6b-41aa-acf8-e6eeff8ca094 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.382828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85319-dc6b-41aa-acf8-e6eeff8ca094 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.383110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca85319-dc6b-41aa-acf8-e6eeff8ca094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.401292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6ca85319-dc6b-41aa-acf8-e6eeff8ca094 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:54.402527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ca85319-dc6b-41aa-acf8-e6eeff8ca094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:54.440395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe40a1f-a8fb-4058-bde8-becc1facc6eb None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:54.446334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:54.448125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.448125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.448125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] PATH_INFO: `/v3/users/8b29fbac1b5142429360db503f2866fa` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.449314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b29fbac1b5142429360db503f2866fa)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:54.451297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:54.469407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1084/2188] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:54.470316 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca85319-dc6b-41aa-acf8-e6eeff8ca094 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:54.471929 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ca85319-dc6b-41aa-acf8-e6eeff8ca094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:54.542684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca85319-dc6b-41aa-acf8-e6eeff8ca094 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:54.547138 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675201c9-3e7a-405f-a543-c069e4e877a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3f786b791b4c8796d251948917ca93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:54.547787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675201c9-3e7a-405f-a543-c069e4e877a5 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.548124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675201c9-3e7a-405f-a543-c069e4e877a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.548352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675201c9-3e7a-405f-a543-c069e4e877a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.550171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-675201c9-3e7a-405f-a543-c069e4e877a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:54.561484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675201c9-3e7a-405f-a543-c069e4e877a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:54.562947 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675201c9-3e7a-405f-a543-c069e4e877a5 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:54.571681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1085/2189] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:54.573131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319392c-4279-48e5-85da-06bca0107e9c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.573418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319392c-4279-48e5-85da-06bca0107e9c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.573695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319392c-4279-48e5-85da-06bca0107e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.591786 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5319392c-4279-48e5-85da-06bca0107e9c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:54.592975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5319392c-4279-48e5-85da-06bca0107e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:54.601412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1086/2190] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:54.605560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319392c-4279-48e5-85da-06bca0107e9c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:54.615833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319392c-4279-48e5-85da-06bca0107e9c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:54.620486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:54.621181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.621582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.621925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.622652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:54.622960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:54.624550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:54.637214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1087/2191] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:54.641399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:54.658985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b863644-28a1-4dd9-825a-df6c4447b091 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:54.663824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b29fbac1b5142429360db503f2866fa'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.664653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b29fbac1b5142429360db503f2866fa'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.665982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a8c1-f959-42d5-8526-20dc0389bea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:54.666637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a8c1-f959-42d5-8526-20dc0389bea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.666928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a8c1-f959-42d5-8526-20dc0389bea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.667280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129a8c1-f959-42d5-8526-20dc0389bea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.668011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129a8c1-f959-42d5-8526-20dc0389bea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:54.670926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129a8c1-f959-42d5-8526-20dc0389bea0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:54.740076 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b29fbac1b5142429360db503f2866fa'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.748885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b29fbac1b5142429360db503f2866fa'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.748885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b29fbac1b5142429360db503f2866fa'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:54.769683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1105/2192] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:54 2022] DELETE /identity/v3/users/8b29fbac1b5142429360db503f2866fa => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:54.783907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1088/2193] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:54 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:54.788327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:54.794991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:54.882017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0eb56-e76a-4ff7-9f5f-1e23ea1f16ca admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:54.943366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7c287-5477-44db-ac6b-96ce47182984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:54.944060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7c287-5477-44db-ac6b-96ce47182984 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:54.944346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7c287-5477-44db-ac6b-96ce47182984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:54.944624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7c287-5477-44db-ac6b-96ce47182984 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:54.945326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7c287-5477-44db-ac6b-96ce47182984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:54.947382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7c287-5477-44db-ac6b-96ce47182984 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.059287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1106/2194] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:54 2022] POST /identity/v3/users => generated 290 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:55.065464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a8c1-f959-42d5-8526-20dc0389bea0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.082180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129a8c1-f959-42d5-8526-20dc0389bea0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.088252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9d230-674c-46a8-beb2-a95976ef288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.091769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9d230-674c-46a8-beb2-a95976ef288c admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.092329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9d230-674c-46a8-beb2-a95976ef288c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.105287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9d230-674c-46a8-beb2-a95976ef288c admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.106679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9d230-674c-46a8-beb2-a95976ef288c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.112498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a9d230-674c-46a8-beb2-a95976ef288c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.125526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1089/2195] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:55.130049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7c287-5477-44db-ac6b-96ce47182984 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.146640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7c287-5477-44db-ac6b-96ce47182984 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.154585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.155536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.156006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.156495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.157503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.159996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.177387 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1107/2196] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:55.178733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9d230-674c-46a8-beb2-a95976ef288c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.190699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9d230-674c-46a8-beb2-a95976ef288c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.195314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6fc5a-03b2-47e2-99de-a01e1efdf543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.196194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6fc5a-03b2-47e2-99de-a01e1efdf543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.196478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6fc5a-03b2-47e2-99de-a01e1efdf543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.196758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6fc5a-03b2-47e2-99de-a01e1efdf543 admin admin] PATH_INFO: `/v3/projects/038242d14b9d498ba84b1d35c4dae139/users/3831f6be9a1c4cc59109efe434d98e25/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.214402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6fc5a-03b2-47e2-99de-a01e1efdf543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=038242d14b9d498ba84b1d35c4dae139, user_id=3831f6be9a1c4cc59109efe434d98e25, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.217538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6fc5a-03b2-47e2-99de-a01e1efdf543 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.279502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.283666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:55.295856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8464a4-4157-4a2d-8f8e-0ff9016afbe0 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.301754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.302831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.303369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.303369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] PATH_INFO: `/v3/projects/5df515c3ca9b447d95509295acbee56c` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.304635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df515c3ca9b447d95509295acbee56c)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.306838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.314389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1090/2197] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:55 2022] PUT /identity/v3/projects/038242d14b9d498ba84b1d35c4dae139/users/3831f6be9a1c4cc59109efe434d98e25/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:55.325493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccc447-f73f-4819-9e68-586064a2a7fa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.325902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccc447-f73f-4819-9e68-586064a2a7fa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.326182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccc447-f73f-4819-9e68-586064a2a7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.361304 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5bccc447-f73f-4819-9e68-586064a2a7fa None None] MFA Rules not processed for user `3831f6be9a1c4cc59109efe434d98e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:55.363049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5bccc447-f73f-4819-9e68-586064a2a7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:55.435373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1091/2198] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:55.439643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccc447-f73f-4819-9e68-586064a2a7fa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.534074 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccc447-f73f-4819-9e68-586064a2a7fa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.535369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df515c3ca9b447d95509295acbee56c', 'request_id': 'req-072ddf1b-7ace-4eb0-a3d0-912755ccc378'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:55.544593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df515c3ca9b447d95509295acbee56c', 'request_id': 'req-072ddf1b-7ace-4eb0-a3d0-912755ccc378'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:55.612372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df515c3ca9b447d95509295acbee56c', 'request_id': 'req-072ddf1b-7ace-4eb0-a3d0-912755ccc378'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:55.617952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.618949 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.619399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.620083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.626675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.626675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.626943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] The token cache is being invalidate because project 5df515c3ca9b447d95509295acbee56c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:55.627069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:55.627887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-072ddf1b-7ace-4eb0-a3d0-912755ccc378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:55.630228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1108/2199] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:55 2022] DELETE /identity/v3/projects/5df515c3ca9b447d95509295acbee56c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:55.635791 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ffe4-957d-496a-bc3f-452252bac493 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.636650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ffe4-957d-496a-bc3f-452252bac493 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.637145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ffe4-957d-496a-bc3f-452252bac493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.657868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bb84ffe4-957d-496a-bc3f-452252bac493 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:55.661330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bb84ffe4-957d-496a-bc3f-452252bac493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:55.669541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1109/2200] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:55.677429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915df2a-77ba-4615-a104-424863613630 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.677729 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915df2a-77ba-4615-a104-424863613630 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.678048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915df2a-77ba-4615-a104-424863613630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.697090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b915df2a-77ba-4615-a104-424863613630 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:55.698570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b915df2a-77ba-4615-a104-424863613630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:55.706559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1110/2201] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:55.711028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915df2a-77ba-4615-a104-424863613630 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.721742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1092/2202] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:55 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:55.726883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915df2a-77ba-4615-a104-424863613630 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.727143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.727143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:55.775924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.776693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.777039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.777384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.778528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:26:55.778628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.780343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.790812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cf1ee-0312-4c34-9cc0-a4f1207066db admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.791900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1111/2203] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:26:55.796479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d25be-8c7a-4450-804b-2038ae2f40b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.796821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.797219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d25be-8c7a-4450-804b-2038ae2f40b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.797512 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d25be-8c7a-4450-804b-2038ae2f40b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.797812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d25be-8c7a-4450-804b-2038ae2f40b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.798554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d25be-8c7a-4450-804b-2038ae2f40b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.800530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d25be-8c7a-4450-804b-2038ae2f40b1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.806856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe62fe1-b5c5-4851-b5bd-57846895789c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.812434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf4d47-ea3f-48d6-b950-08f45e6b2feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.813099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf4d47-ea3f-48d6-b950-08f45e6b2feb admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.813517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf4d47-ea3f-48d6-b950-08f45e6b2feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.813831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf4d47-ea3f-48d6-b950-08f45e6b2feb admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.814535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf4d47-ea3f-48d6-b950-08f45e6b2feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.816459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf4d47-ea3f-48d6-b950-08f45e6b2feb admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.913971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1093/2204] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:55 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:55.918026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d25be-8c7a-4450-804b-2038ae2f40b1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.918273 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1112/2205] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:55 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:55.928975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d25be-8c7a-4450-804b-2038ae2f40b1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.931549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf4d47-ea3f-48d6-b950-08f45e6b2feb admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:55.988148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285370f8-72ef-400c-95ce-39748a2751a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.988991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285370f8-72ef-400c-95ce-39748a2751a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.989378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285370f8-72ef-400c-95ce-39748a2751a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:55.989763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285370f8-72ef-400c-95ce-39748a2751a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:55.990631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285370f8-72ef-400c-95ce-39748a2751a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:55.992352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285370f8-72ef-400c-95ce-39748a2751a1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:55.993724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf4d47-ea3f-48d6-b950-08f45e6b2feb admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:55.998943 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650084-fa96-4d84-ac9f-3f0e1d13ef68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:55.999608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650084-fa96-4d84-ac9f-3f0e1d13ef68 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:55.999892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650084-fa96-4d84-ac9f-3f0e1d13ef68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.000169 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650084-fa96-4d84-ac9f-3f0e1d13ef68 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.000971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a650084-fa96-4d84-ac9f-3f0e1d13ef68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.003267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1094/2206] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:56.003388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a650084-fa96-4d84-ac9f-3f0e1d13ef68 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.007187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285370f8-72ef-400c-95ce-39748a2751a1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.017777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285370f8-72ef-400c-95ce-39748a2751a1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.023317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdc200-61fc-4da9-9960-41e274a19b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.024213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdc200-61fc-4da9-9960-41e274a19b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.024483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdc200-61fc-4da9-9960-41e274a19b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.024758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdc200-61fc-4da9-9960-41e274a19b4b admin admin] PATH_INFO: `/v3/projects/fe7a6650a4df49cdb8c7e0efa92da041/users/6982c3c5e7d543b5b5c2864636ef5745/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.040416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdc200-61fc-4da9-9960-41e274a19b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe7a6650a4df49cdb8c7e0efa92da041, user_id=6982c3c5e7d543b5b5c2864636ef5745, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.043192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdc200-61fc-4da9-9960-41e274a19b4b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.080006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1113/2207] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:55 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:26:56.083872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650084-fa96-4d84-ac9f-3f0e1d13ef68 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.093980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650084-fa96-4d84-ac9f-3f0e1d13ef68 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.099124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259524e0-b70f-4bda-89df-ede1232bd166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.099848 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259524e0-b70f-4bda-89df-ede1232bd166 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.100134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259524e0-b70f-4bda-89df-ede1232bd166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.100412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259524e0-b70f-4bda-89df-ede1232bd166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.101141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259524e0-b70f-4bda-89df-ede1232bd166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.102864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259524e0-b70f-4bda-89df-ede1232bd166 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.107256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1095/2208] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:56 2022] PUT /identity/v3/projects/fe7a6650a4df49cdb8c7e0efa92da041/users/6982c3c5e7d543b5b5c2864636ef5745/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:56.109961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdc200-61fc-4da9-9960-41e274a19b4b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.119496 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1114/2209] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:56.120541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259524e0-b70f-4bda-89df-ede1232bd166 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.131425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259524e0-b70f-4bda-89df-ede1232bd166 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.180727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdc200-61fc-4da9-9960-41e274a19b4b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.185499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08c7a0-7904-4fd3-b4da-3f0c5f786b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.186236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c7a0-7904-4fd3-b4da-3f0c5f786b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.186522 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c7a0-7904-4fd3-b4da-3f0c5f786b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.186801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08c7a0-7904-4fd3-b4da-3f0c5f786b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.187537 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08c7a0-7904-4fd3-b4da-3f0c5f786b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.189149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08c7a0-7904-4fd3-b4da-3f0c5f786b3a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.193747 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1487d-acb7-4610-9ee7-9aa14aa7061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.194595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1487d-acb7-4610-9ee7-9aa14aa7061c admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.194947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1487d-acb7-4610-9ee7-9aa14aa7061c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.195289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1487d-acb7-4610-9ee7-9aa14aa7061c admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.196086 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1487d-acb7-4610-9ee7-9aa14aa7061c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.198433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1487d-acb7-4610-9ee7-9aa14aa7061c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.199912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1096/2210] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:56.204126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08c7a0-7904-4fd3-b4da-3f0c5f786b3a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.209816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1115/2211] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:26:56.213472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1487d-acb7-4610-9ee7-9aa14aa7061c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.214765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08c7a0-7904-4fd3-b4da-3f0c5f786b3a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.219503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9473c38-1ac9-480a-8773-afdab8498cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.220375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9473c38-1ac9-480a-8773-afdab8498cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.220659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9473c38-1ac9-480a-8773-afdab8498cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.220935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9473c38-1ac9-480a-8773-afdab8498cfd admin admin] PATH_INFO: `/v3/projects/fe7a6650a4df49cdb8c7e0efa92da041/users/6982c3c5e7d543b5b5c2864636ef5745/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.222986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9473c38-1ac9-480a-8773-afdab8498cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe7a6650a4df49cdb8c7e0efa92da041, user_id=6982c3c5e7d543b5b5c2864636ef5745, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.224879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1487d-acb7-4610-9ee7-9aa14aa7061c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.225706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9473c38-1ac9-480a-8773-afdab8498cfd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.229975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7fd7d-fe2d-4f3b-86c6-eca03f0be601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.231130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fd7d-fe2d-4f3b-86c6-eca03f0be601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.232165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fd7d-fe2d-4f3b-86c6-eca03f0be601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.232368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fd7d-fe2d-4f3b-86c6-eca03f0be601 admin admin] PATH_INFO: `/v3/projects/5859fef6ef91481e985ec7ee3e6f80d4/users/191081c494214365b5dead54e0a67ec9/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.237635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1097/2212] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:56 2022] PUT /identity/v3/projects/fe7a6650a4df49cdb8c7e0efa92da041/users/6982c3c5e7d543b5b5c2864636ef5745/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:56.249289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fcaa7-640f-4aba-a3a0-8f1c32d971dc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.249849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fcaa7-640f-4aba-a3a0-8f1c32d971dc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.250595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fcaa7-640f-4aba-a3a0-8f1c32d971dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.250992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7fd7d-fe2d-4f3b-86c6-eca03f0be601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5859fef6ef91481e985ec7ee3e6f80d4, user_id=191081c494214365b5dead54e0a67ec9, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.253756 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7fd7d-fe2d-4f3b-86c6-eca03f0be601 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.280080 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-487fcaa7-640f-4aba-a3a0-8f1c32d971dc None None] MFA Rules not processed for user `6982c3c5e7d543b5b5c2864636ef5745`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:56.281395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-487fcaa7-640f-4aba-a3a0-8f1c32d971dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:56.349837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1116/2213] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:56 2022] PUT /identity/v3/projects/5859fef6ef91481e985ec7ee3e6f80d4/users/191081c494214365b5dead54e0a67ec9/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:26:56.361244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39bc45-0d90-4ddc-b504-200db60ac5f5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.361443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39bc45-0d90-4ddc-b504-200db60ac5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.361727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39bc45-0d90-4ddc-b504-200db60ac5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.365811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1098/2214] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:56.376926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fcaa7-640f-4aba-a3a0-8f1c32d971dc None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.378214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-487fcaa7-640f-4aba-a3a0-8f1c32d971dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:56.395411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ea39bc45-0d90-4ddc-b504-200db60ac5f5 None None] MFA Rules not processed for user `191081c494214365b5dead54e0a67ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:56.396783 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ea39bc45-0d90-4ddc-b504-200db60ac5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:56.442014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fcaa7-640f-4aba-a3a0-8f1c32d971dc None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.472266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1117/2215] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:26:56.479589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39bc45-0d90-4ddc-b504-200db60ac5f5 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.515008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58aa894-1394-4156-864c-ed6273345dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.515662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58aa894-1394-4156-864c-ed6273345dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.515947 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58aa894-1394-4156-864c-ed6273345dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.516360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58aa894-1394-4156-864c-ed6273345dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.563877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39bc45-0d90-4ddc-b504-200db60ac5f5 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.571177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ed0e7-47a5-4a36-bf7a-c124a4148d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.572161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed0e7-47a5-4a36-bf7a-c124a4148d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.572438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed0e7-47a5-4a36-bf7a-c124a4148d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.573126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed0e7-47a5-4a36-bf7a-c124a4148d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.573868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ed0e7-47a5-4a36-bf7a-c124a4148d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.577140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ed0e7-47a5-4a36-bf7a-c124a4148d09 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.603387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58aa894-1394-4156-864c-ed6273345dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.604845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58aa894-1394-4156-864c-ed6273345dc7 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.615782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1099/2216] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:56.685429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1118/2217] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:26:56 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:56.686956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58aa894-1394-4156-864c-ed6273345dc7 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.698608 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58aa894-1394-4156-864c-ed6273345dc7 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.768836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0236cc-3bc1-424a-b0e1-82b6445ca0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.769556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0236cc-3bc1-424a-b0e1-82b6445ca0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.769912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0236cc-3bc1-424a-b0e1-82b6445ca0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.770183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0236cc-3bc1-424a-b0e1-82b6445ca0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.770927 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0236cc-3bc1-424a-b0e1-82b6445ca0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.772861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0236cc-3bc1-424a-b0e1-82b6445ca0f3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.796224 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ed0e7-47a5-4a36-bf7a-c124a4148d09 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.864037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ed0e7-47a5-4a36-bf7a-c124a4148d09 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.868324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.868602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.868886 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.869162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.871924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:56.885295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1100/2218] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:26:56 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:26:56.889297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0236cc-3bc1-424a-b0e1-82b6445ca0f3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.899097 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0236cc-3bc1-424a-b0e1-82b6445ca0f3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.906824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57327317-1fe7-45ce-b437-9163fe7a65aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.906824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57327317-1fe7-45ce-b437-9163fe7a65aa admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.906824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57327317-1fe7-45ce-b437-9163fe7a65aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.906824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57327317-1fe7-45ce-b437-9163fe7a65aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.907248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57327317-1fe7-45ce-b437-9163fe7a65aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.908358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57327317-1fe7-45ce-b437-9163fe7a65aa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.919537 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1101/2219] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:56.924215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57327317-1fe7-45ce-b437-9163fe7a65aa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:56.936169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57327317-1fe7-45ce-b437-9163fe7a65aa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:56.941347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47521427-f68e-417b-8a2b-dc687bd48554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:56.942279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521427-f68e-417b-8a2b-dc687bd48554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:56.942599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.942714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521427-f68e-417b-8a2b-dc687bd48554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:56.942946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521427-f68e-417b-8a2b-dc687bd48554 admin admin] PATH_INFO: `/v3/projects/faa5c8ed083c49ae991ab96fd680f5cd/users/5dce473fd96a455fa8a1e3acc9615e00/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:56.945573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:56.960471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1119/2220] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:26:56.961775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47521427-f68e-417b-8a2b-dc687bd48554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa5c8ed083c49ae991ab96fd680f5cd, user_id=5dce473fd96a455fa8a1e3acc9615e00, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:56.964571 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47521427-f68e-417b-8a2b-dc687bd48554 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:57.067187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1102/2221] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:56 2022] PUT /identity/v3/projects/faa5c8ed083c49ae991ab96fd680f5cd/users/5dce473fd96a455fa8a1e3acc9615e00/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:57.071303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:57.087533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ff51-d0f1-4b86-adf7-981699a59651 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:57.159300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9add95f-1d17-47d9-a791-1e1f65f00984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:57.159974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9add95f-1d17-47d9-a791-1e1f65f00984 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:57.160265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9add95f-1d17-47d9-a791-1e1f65f00984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:57.160532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9add95f-1d17-47d9-a791-1e1f65f00984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:57.161257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9add95f-1d17-47d9-a791-1e1f65f00984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:57.162888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9add95f-1d17-47d9-a791-1e1f65f00984 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:57.173854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1120/2222] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:26:57.178078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47521427-f68e-417b-8a2b-dc687bd48554 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:57.188869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47521427-f68e-417b-8a2b-dc687bd48554 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:57.193846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:57.194717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:57.194996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:57.195269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] PATH_INFO: `/v3/projects/faa5c8ed083c49ae991ab96fd680f5cd/users/5dce473fd96a455fa8a1e3acc9615e00/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:57.197244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa5c8ed083c49ae991ab96fd680f5cd, user_id=5dce473fd96a455fa8a1e3acc9615e00, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:57.200185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:57.210329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1103/2223] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:26:57 2022] PUT /identity/v3/projects/faa5c8ed083c49ae991ab96fd680f5cd/users/5dce473fd96a455fa8a1e3acc9615e00/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:57.223951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c29d-e95d-4b47-96d4-65bb438a8cd7 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:57.223951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c29d-e95d-4b47-96d4-65bb438a8cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:57.223951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c29d-e95d-4b47-96d4-65bb438a8cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:57.263773 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-acf4c29d-e95d-4b47-96d4-65bb438a8cd7 None None] MFA Rules not processed for user `5dce473fd96a455fa8a1e3acc9615e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:57.264987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-acf4c29d-e95d-4b47-96d4-65bb438a8cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:57.337002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1121/2224] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:57.348725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:57.353009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:57.430349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dfbe5-e528-4f7b-be06-14a08ed26431 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:57.435236 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e29ce-461f-466e-8064-4654cac9abe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:57.435888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e29ce-461f-466e-8064-4654cac9abe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:57.436171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e29ce-461f-466e-8064-4654cac9abe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:57.436441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e29ce-461f-466e-8064-4654cac9abe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:57.446015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e29ce-461f-466e-8064-4654cac9abe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:57.447535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e29ce-461f-466e-8064-4654cac9abe5 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:57.457361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1104/2225] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:58.035151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4c29d-e95d-4b47-96d4-65bb438a8cd7 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:58.095815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4c29d-e95d-4b47-96d4-65bb438a8cd7 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:58.100475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:58.101353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:58.101637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:58.101946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] PATH_INFO: `/v3/users/191081c494214365b5dead54e0a67ec9` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:58.103190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=191081c494214365b5dead54e0a67ec9)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:58.105179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:58.340079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '191081c494214365b5dead54e0a67ec9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:58.340733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '191081c494214365b5dead54e0a67ec9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:58.428022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '191081c494214365b5dead54e0a67ec9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:58.435309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '191081c494214365b5dead54e0a67ec9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:58.436425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '191081c494214365b5dead54e0a67ec9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:58.455130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1122/2226] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:58 2022] DELETE /identity/v3/users/191081c494214365b5dead54e0a67ec9 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:58.459863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dfdbb-2d06-4265-9b44-5d3b44825077 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:58.460412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dfdbb-2d06-4265-9b44-5d3b44825077 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:58.460514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dfdbb-2d06-4265-9b44-5d3b44825077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:58.478435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c73dfdbb-2d06-4265-9b44-5d3b44825077 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:26:58.479655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c73dfdbb-2d06-4265-9b44-5d3b44825077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:58.538688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1105/2227] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:26:58.549385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:58.550787 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:58.606099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75266-3bc3-4443-b8cb-4c050b3d4489 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:58.610292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5922e57-4c41-414c-9d68-c86a255d4304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:58.610932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5922e57-4c41-414c-9d68-c86a255d4304 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:58.611221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5922e57-4c41-414c-9d68-c86a255d4304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:58.611500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5922e57-4c41-414c-9d68-c86a255d4304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:58.620247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5922e57-4c41-414c-9d68-c86a255d4304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:58.621728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5922e57-4c41-414c-9d68-c86a255d4304 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:58.631491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1123/2228] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:26:59.176833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dfdbb-2d06-4265-9b44-5d3b44825077 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:59.178612 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c73dfdbb-2d06-4265-9b44-5d3b44825077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:26:59.196146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dfdbb-2d06-4265-9b44-5d3b44825077 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:59.201295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:59.202475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:59.202756 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:59.203029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] PATH_INFO: `/v3/projects/5859fef6ef91481e985ec7ee3e6f80d4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:59.204241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5859fef6ef91481e985ec7ee3e6f80d4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:59.206223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:59.360671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5859fef6ef91481e985ec7ee3e6f80d4', 'request_id': 'req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.366472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5859fef6ef91481e985ec7ee3e6f80d4', 'request_id': 'req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.465810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5859fef6ef91481e985ec7ee3e6f80d4', 'request_id': 'req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.481974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] The token cache is being invalidate because project 5859fef6ef91481e985ec7ee3e6f80d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:26:59.482455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.483355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.484952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1106/2229] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:26:59 2022] DELETE /identity/v3/projects/5859fef6ef91481e985ec7ee3e6f80d4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:26:59.489003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5922e57-4c41-414c-9d68-c86a255d4304 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:26:59.501010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5922e57-4c41-414c-9d68-c86a255d4304 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:26:59.569284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:26:59.570301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:26:59.570585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:26:59.570859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] PATH_INFO: `/v3/users/5dce473fd96a455fa8a1e3acc9615e00` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:26:59.572092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dce473fd96a455fa8a1e3acc9615e00)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:26:59.574434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:26:59.799106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dce473fd96a455fa8a1e3acc9615e00'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.799758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dce473fd96a455fa8a1e3acc9615e00'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.874417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dce473fd96a455fa8a1e3acc9615e00'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.881769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dce473fd96a455fa8a1e3acc9615e00'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.882461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dce473fd96a455fa8a1e3acc9615e00'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:26:59.899802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1124/2230] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:26:59 2022] DELETE /identity/v3/users/5dce473fd96a455fa8a1e3acc9615e00 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:00.222421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:00.225263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:00.246689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a9a21-1733-4fcb-b01c-9c178c89e7ee admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:00.318463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:00.319457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:00.319847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:00.320225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] PATH_INFO: `/v3/users/3831f6be9a1c4cc59109efe434d98e25` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:00.321687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3831f6be9a1c4cc59109efe434d98e25)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:00.324191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:00.402620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:00.404262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:00.423519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffc82-7341-4d3e-a779-783454c39c9f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:00.440040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:00.441077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:00.441375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:00.441658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] PATH_INFO: `/v3/projects/faa5c8ed083c49ae991ab96fd680f5cd` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:00.443303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa5c8ed083c49ae991ab96fd680f5cd)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:00.445353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:00.544942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3831f6be9a1c4cc59109efe434d98e25'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.547107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3831f6be9a1c4cc59109efe434d98e25'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.547920 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3831f6be9a1c4cc59109efe434d98e25'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.564858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3831f6be9a1c4cc59109efe434d98e25'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.599085 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa5c8ed083c49ae991ab96fd680f5cd', 'request_id': 'req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.604433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa5c8ed083c49ae991ab96fd680f5cd', 'request_id': 'req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.677521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-92475135-31b6-4d2f-9d10-6212f4bdfbeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3831f6be9a1c4cc59109efe434d98e25'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.696607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1107/2231] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:27:00 2022] DELETE /identity/v3/users/3831f6be9a1c4cc59109efe434d98e25 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:27:00.701357 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac13c21-3b52-484c-ab4e-40b4ddd31ddc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:00.701658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac13c21-3b52-484c-ab4e-40b4ddd31ddc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:00.701970 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac13c21-3b52-484c-ab4e-40b4ddd31ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:00.721229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cac13c21-3b52-484c-ab4e-40b4ddd31ddc None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:00.722482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cac13c21-3b52-484c-ab4e-40b4ddd31ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:00.774126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa5c8ed083c49ae991ab96fd680f5cd', 'request_id': 'req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.784219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1108/2232] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:00.786632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] The token cache is being invalidate because project faa5c8ed083c49ae991ab96fd680f5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:27:00.786925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.787566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:00.789092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1125/2233] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:27:00 2022] DELETE /identity/v3/projects/faa5c8ed083c49ae991ab96fd680f5cd => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:00.793943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8ff03-6a86-4ddf-a781-feea0df58ed4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:00.794072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:00.794546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8ff03-6a86-4ddf-a781-feea0df58ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:00.794833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8ff03-6a86-4ddf-a781-feea0df58ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:00.796164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:00.821631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c1e8ff03-6a86-4ddf-a781-feea0df58ed4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:00.823461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c1e8ff03-6a86-4ddf-a781-feea0df58ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:00.837473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1109/2234] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:00.851367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866a323-8be2-413c-9361-8d5281593ec4 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:00.851982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866a323-8be2-413c-9361-8d5281593ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:00.852459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866a323-8be2-413c-9361-8d5281593ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:00.874611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0866a323-8be2-413c-9361-8d5281593ec4 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:00.875574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078b1f4-8f5f-44cd-bac3-6a88c1db546c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:00.876018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0866a323-8be2-413c-9361-8d5281593ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:00.888375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1110/2235] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:00.893633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866a323-8be2-413c-9361-8d5281593ec4 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:00.905563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866a323-8be2-413c-9361-8d5281593ec4 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:00.917087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:00.918437 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:00.919022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:00.919482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:00.920468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:27:00.921581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:00.923842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:00.934678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:00.935391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:00.935705 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:00.936008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:00.941942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:00.942076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1111/2236] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:27:00.944898 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:00.953395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:00.955663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:00.959479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b52036-95ad-43e0-aa03-21ad98ba56eb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:00.966478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21072b28-b22a-45f4-8583-90b243ac77fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:00.970606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072b28-b22a-45f4-8583-90b243ac77fe admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:00.970606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072b28-b22a-45f4-8583-90b243ac77fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:00.970606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072b28-b22a-45f4-8583-90b243ac77fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:00.970606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21072b28-b22a-45f4-8583-90b243ac77fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:00.971901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21072b28-b22a-45f4-8583-90b243ac77fe admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:00.981396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1126/2237] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:27:01.095573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1112/2238] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:27:00 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:27:01.101152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:01.113957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdfea4-7cdd-4004-9e31-4a6c43e252fe service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:01.216699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b87368-3fab-4784-9e4e-55bb3117a229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:01.218117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87368-3fab-4784-9e4e-55bb3117a229 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.218905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87368-3fab-4784-9e4e-55bb3117a229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.219502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87368-3fab-4784-9e4e-55bb3117a229 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.220756 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b87368-3fab-4784-9e4e-55bb3117a229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:01.222902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b87368-3fab-4784-9e4e-55bb3117a229 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:01.389008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1127/2239] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:27:01 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:27:01.393502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21072b28-b22a-45f4-8583-90b243ac77fe admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:01.406639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21072b28-b22a-45f4-8583-90b243ac77fe admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:01.412218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d4b27-eae9-4a97-88a0-012381747629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:01.413044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d4b27-eae9-4a97-88a0-012381747629 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.413336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d4b27-eae9-4a97-88a0-012381747629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.413605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d4b27-eae9-4a97-88a0-012381747629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.415361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d4b27-eae9-4a97-88a0-012381747629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:01.426629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d4b27-eae9-4a97-88a0-012381747629 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:01.440888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1113/2240] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:27:01.442578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b87368-3fab-4784-9e4e-55bb3117a229 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:01.463539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b87368-3fab-4784-9e4e-55bb3117a229 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:01.469774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:01.470567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.470942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.471302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.472151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:01.474008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:01.487288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1128/2241] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:27:01.491453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d4b27-eae9-4a97-88a0-012381747629 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:01.501997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d4b27-eae9-4a97-88a0-012381747629 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:01.506622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89deb199-6374-4b8c-8fb4-df6ab5f23143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:01.507542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deb199-6374-4b8c-8fb4-df6ab5f23143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.507825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deb199-6374-4b8c-8fb4-df6ab5f23143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.508100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deb199-6374-4b8c-8fb4-df6ab5f23143 admin admin] PATH_INFO: `/v3/projects/3c2ba40cbebd4c1db39ca98fbd9bff0a/users/d948db141ee8444d996233b57eb545f8/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.521641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:01.523909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:01.526064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89deb199-6374-4b8c-8fb4-df6ab5f23143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2ba40cbebd4c1db39ca98fbd9bff0a, user_id=d948db141ee8444d996233b57eb545f8, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:01.537994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89deb199-6374-4b8c-8fb4-df6ab5f23143 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:01.539971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d17f19-0cf7-44e6-a463-f8ac086d3504 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:01.545948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:01.546834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.547145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.547425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] PATH_INFO: `/v3/projects/038242d14b9d498ba84b1d35c4dae139` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.548864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=038242d14b9d498ba84b1d35c4dae139)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:01.550855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:01.622979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1114/2242] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:27:01 2022] PUT /identity/v3/projects/3c2ba40cbebd4c1db39ca98fbd9bff0a/users/d948db141ee8444d996233b57eb545f8/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:27:01.633094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07bcc-65af-4cb2-8f85-8f661af7a3ff None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.633377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07bcc-65af-4cb2-8f85-8f661af7a3ff None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.633651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07bcc-65af-4cb2-8f85-8f661af7a3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.664053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038242d14b9d498ba84b1d35c4dae139', 'request_id': 'req-20b46540-17cd-45af-aa0f-7d5b394d1a0c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:01.664640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6aa07bcc-65af-4cb2-8f85-8f661af7a3ff None None] MFA Rules not processed for user `d948db141ee8444d996233b57eb545f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:01.665360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6aa07bcc-65af-4cb2-8f85-8f661af7a3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:01.670564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038242d14b9d498ba84b1d35c4dae139', 'request_id': 'req-20b46540-17cd-45af-aa0f-7d5b394d1a0c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:01.733134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1115/2243] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:01.744063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07bcc-65af-4cb2-8f85-8f661af7a3ff None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:01.745406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6aa07bcc-65af-4cb2-8f85-8f661af7a3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:01.748225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038242d14b9d498ba84b1d35c4dae139', 'request_id': 'req-20b46540-17cd-45af-aa0f-7d5b394d1a0c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:01.760077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] The token cache is being invalidate because project 038242d14b9d498ba84b1d35c4dae139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:27:01.760367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:01.761003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:01.762589 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1129/2244] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:27:01 2022] DELETE /identity/v3/projects/038242d14b9d498ba84b1d35c4dae139 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:01.767156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:01.768596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:01.804570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07bcc-65af-4cb2-8f85-8f661af7a3ff None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:01.806849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:01.807515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.807785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.808062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.809862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:01.820784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:01.822282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:01.832708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b46540-17cd-45af-aa0f-7d5b394d1a0c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:01.840186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1116/2245] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:27:01.841262 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:01.842775 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.843063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.843520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] PATH_INFO: `/v3/users/6982c3c5e7d543b5b5c2864636ef5745` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.844759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6982c3c5e7d543b5b5c2864636ef5745)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:01.846797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:01.865660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:01.867458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:01.881407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e551f0-86d9-4b57-bf90-300f8794c491 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:01.886290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:01.887167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:01.887478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:01.887779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] PATH_INFO: `/v3/users/d948db141ee8444d996233b57eb545f8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:01.889564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d948db141ee8444d996233b57eb545f8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:01.891535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:02.085986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6982c3c5e7d543b5b5c2864636ef5745'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.086605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6982c3c5e7d543b5b5c2864636ef5745'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.095890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd948db141ee8444d996233b57eb545f8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.096548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd948db141ee8444d996233b57eb545f8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.097137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd948db141ee8444d996233b57eb545f8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.103654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd948db141ee8444d996233b57eb545f8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.168427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6982c3c5e7d543b5b5c2864636ef5745'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.176504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6982c3c5e7d543b5b5c2864636ef5745'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.177150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f3a5530a-ccd2-4f89-84c2-d79bfa5f68d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6982c3c5e7d543b5b5c2864636ef5745'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.196938 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1130/2246] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:27:01 2022] DELETE /identity/v3/users/6982c3c5e7d543b5b5c2864636ef5745 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:27:02.239898 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd948db141ee8444d996233b57eb545f8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:02.259024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1117/2247] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:27:01 2022] DELETE /identity/v3/users/d948db141ee8444d996233b57eb545f8 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:27:02.263560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70be248-5a14-42d4-b87e-b40bf621921f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:02.264600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70be248-5a14-42d4-b87e-b40bf621921f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:02.265054 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70be248-5a14-42d4-b87e-b40bf621921f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:02.282757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f70be248-5a14-42d4-b87e-b40bf621921f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:02.284119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f70be248-5a14-42d4-b87e-b40bf621921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:02.350295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1131/2248] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:02.359826 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:02.361249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:02.422863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafd804-bf90-4ae5-86c6-f4d1650c514f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:02.426641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:02.427359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:02.427637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:02.427914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:02.437665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:02.439141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:02.448780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1118/2249] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:27:02.910574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70be248-5a14-42d4-b87e-b40bf621921f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:02.912251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f70be248-5a14-42d4-b87e-b40bf621921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:02.947485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70be248-5a14-42d4-b87e-b40bf621921f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:02.953735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:02.954750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:02.955132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:02.955506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] PATH_INFO: `/v3/projects/fe7a6650a4df49cdb8c7e0efa92da041` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:02.957217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe7a6650a4df49cdb8c7e0efa92da041)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:02.959716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.028628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.031773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:03.047164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5436a02-0179-4ab3-9740-ab0a86c90586 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.051935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.052873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.053233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.053579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] PATH_INFO: `/v3/projects/3c2ba40cbebd4c1db39ca98fbd9bff0a` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.055161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c2ba40cbebd4c1db39ca98fbd9bff0a)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.057647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.143674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe7a6650a4df49cdb8c7e0efa92da041', 'request_id': 'req-a5edb4df-43fb-46ea-b8c6-3f311888099d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.149108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe7a6650a4df49cdb8c7e0efa92da041', 'request_id': 'req-a5edb4df-43fb-46ea-b8c6-3f311888099d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.169028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c2ba40cbebd4c1db39ca98fbd9bff0a', 'request_id': 'req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.174904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c2ba40cbebd4c1db39ca98fbd9bff0a', 'request_id': 'req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.228299 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe7a6650a4df49cdb8c7e0efa92da041', 'request_id': 'req-a5edb4df-43fb-46ea-b8c6-3f311888099d'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.242689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] The token cache is being invalidate because project fe7a6650a4df49cdb8c7e0efa92da041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:27:03.242985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.246898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a5edb4df-43fb-46ea-b8c6-3f311888099d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.249331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1132/2250] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:27:02 2022] DELETE /identity/v3/projects/fe7a6650a4df49cdb8c7e0efa92da041 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:03.254087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6d10-4a6e-4702-8486-2e3f7a44fb4d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.254422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6d10-4a6e-4702-8486-2e3f7a44fb4d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.254721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6d10-4a6e-4702-8486-2e3f7a44fb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.273998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c2ba40cbebd4c1db39ca98fbd9bff0a', 'request_id': 'req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.277478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-9a5b6d10-4a6e-4702-8486-2e3f7a44fb4d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:03.278891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9a5b6d10-4a6e-4702-8486-2e3f7a44fb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:03.286526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] The token cache is being invalidate because project 3c2ba40cbebd4c1db39ca98fbd9bff0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:27:03.286812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.287476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ba3e563f-e7f1-464b-ab2e-15cc0ecd2b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:03.289062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1119/2251] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:27:03 2022] DELETE /identity/v3/projects/3c2ba40cbebd4c1db39ca98fbd9bff0a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:27:03.295995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989ee66-1d7e-4d60-a14e-494138edcfdc None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.296282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989ee66-1d7e-4d60-a14e-494138edcfdc None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.296585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989ee66-1d7e-4d60-a14e-494138edcfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.315766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5989ee66-1d7e-4d60-a14e-494138edcfdc None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:03.317070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5989ee66-1d7e-4d60-a14e-494138edcfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:03.345464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1133/2252] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:03.353385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950fa1-9a62-4e0d-b12d-063292a09d96 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.353680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950fa1-9a62-4e0d-b12d-063292a09d96 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.353997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950fa1-9a62-4e0d-b12d-063292a09d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.373027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2f950fa1-9a62-4e0d-b12d-063292a09d96 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:03.374343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2f950fa1-9a62-4e0d-b12d-063292a09d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:03.378614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1120/2253] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:03.387330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1134/2254] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:03.389586 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7043e7b-862d-4d07-b235-e7bdf14e9817 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.390106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7043e7b-862d-4d07-b235-e7bdf14e9817 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.390257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7043e7b-862d-4d07-b235-e7bdf14e9817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.390987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f950fa1-9a62-4e0d-b12d-063292a09d96 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.400205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f950fa1-9a62-4e0d-b12d-063292a09d96 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.405835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.406503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.406808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.407081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.407804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:27:03.408108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.409168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e7043e7b-862d-4d07-b235-e7bdf14e9817 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:03.411046 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.411161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e7043e7b-862d-4d07-b235-e7bdf14e9817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:03.421104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1121/2255] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:03.422369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1135/2256] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:27:03.427293 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7043e7b-862d-4d07-b235-e7bdf14e9817 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.427478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.437098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7043e7b-862d-4d07-b235-e7bdf14e9817 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.438260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e66ee8-748b-4bb4-bbb0-b569a1f16322 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.441890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.442562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.442847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.443125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.443323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918794e5-073c-4097-95d3-534d7a01f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.443866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:27:03.443981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918794e5-073c-4097-95d3-534d7a01f4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.444241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918794e5-073c-4097-95d3-534d7a01f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.444343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.444508 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918794e5-073c-4097-95d3-534d7a01f4e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.445193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918794e5-073c-4097-95d3-534d7a01f4e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.445928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.447118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918794e5-073c-4097-95d3-534d7a01f4e5 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.459125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1122/2257] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:27:03.470891 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.479689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ae02d-14ff-4a14-88f2-99bcaf1cf5b6 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.486217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01197c52-3a6a-4dcb-951c-9aeb67062843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.487225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01197c52-3a6a-4dcb-951c-9aeb67062843 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.487854 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01197c52-3a6a-4dcb-951c-9aeb67062843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.488138 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01197c52-3a6a-4dcb-951c-9aeb67062843 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.488865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01197c52-3a6a-4dcb-951c-9aeb67062843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.490850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01197c52-3a6a-4dcb-951c-9aeb67062843 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.567487 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1136/2258] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:27:03 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:27:03.571502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918794e5-073c-4097-95d3-534d7a01f4e5 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.595657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1123/2259] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:27:03 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:27:03.600457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01197c52-3a6a-4dcb-951c-9aeb67062843 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.641511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918794e5-073c-4097-95d3-534d7a01f4e5 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.670053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01197c52-3a6a-4dcb-951c-9aeb67062843 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.675973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12eda16-12dc-44a2-a852-02db128773b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.677824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eda16-12dc-44a2-a852-02db128773b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.677824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eda16-12dc-44a2-a852-02db128773b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.677824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eda16-12dc-44a2-a852-02db128773b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.680531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12eda16-12dc-44a2-a852-02db128773b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.680769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12eda16-12dc-44a2-a852-02db128773b3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.711991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdc4a0-938d-4ef5-aa89-3fd252085457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.712964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdc4a0-938d-4ef5-aa89-3fd252085457 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.713319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdc4a0-938d-4ef5-aa89-3fd252085457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.713605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdc4a0-938d-4ef5-aa89-3fd252085457 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.714444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdc4a0-938d-4ef5-aa89-3fd252085457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.716934 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdc4a0-938d-4ef5-aa89-3fd252085457 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.788759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1124/2260] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:27:03 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:27:03.792576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12eda16-12dc-44a2-a852-02db128773b3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.811515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12eda16-12dc-44a2-a852-02db128773b3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.817916 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1137/2261] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:27:03 2022] POST /identity/v3/users => generated 292 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:27:03.821649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdeaf9-bbdf-4cc5-bbe4-c35fdc7940c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.822480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdeaf9-bbdf-4cc5-bbe4-c35fdc7940c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.823550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdeaf9-bbdf-4cc5-bbe4-c35fdc7940c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.826942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdc4a0-938d-4ef5-aa89-3fd252085457 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.827539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdeaf9-bbdf-4cc5-bbe4-c35fdc7940c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.833300 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdeaf9-bbdf-4cc5-bbe4-c35fdc7940c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.835829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdeaf9-bbdf-4cc5-bbe4-c35fdc7940c8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.841149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdc4a0-938d-4ef5-aa89-3fd252085457 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.847970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22400f0e-8415-4695-b19a-2e7bbe7bd0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.855310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400f0e-8415-4695-b19a-2e7bbe7bd0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.855310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400f0e-8415-4695-b19a-2e7bbe7bd0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.855310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22400f0e-8415-4695-b19a-2e7bbe7bd0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.855310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22400f0e-8415-4695-b19a-2e7bbe7bd0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.855310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22400f0e-8415-4695-b19a-2e7bbe7bd0e0 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.855808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1125/2262] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:27:03.858209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdeaf9-bbdf-4cc5-bbe4-c35fdc7940c8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.866718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1138/2263] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:27:03.870378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22400f0e-8415-4695-b19a-2e7bbe7bd0e0 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.871169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdeaf9-bbdf-4cc5-bbe4-c35fdc7940c8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.876710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a832d-7266-4a1e-848c-9950bd53c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.877494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a832d-7266-4a1e-848c-9950bd53c70d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.877878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a832d-7266-4a1e-848c-9950bd53c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.878225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a832d-7266-4a1e-848c-9950bd53c70d admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.879034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a832d-7266-4a1e-848c-9950bd53c70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.884037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22400f0e-8415-4695-b19a-2e7bbe7bd0e0 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.884165 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a832d-7266-4a1e-848c-9950bd53c70d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.885550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe2141-38a7-4d01-834e-31051ec06a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.886381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe2141-38a7-4d01-834e-31051ec06a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.886710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe2141-38a7-4d01-834e-31051ec06a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.886990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe2141-38a7-4d01-834e-31051ec06a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.887864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe2141-38a7-4d01-834e-31051ec06a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.889928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe2141-38a7-4d01-834e-31051ec06a26 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.893828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1126/2264] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:27:03.896133 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a832d-7266-4a1e-848c-9950bd53c70d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.911071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a832d-7266-4a1e-848c-9950bd53c70d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.917560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252e655-317a-4ee2-a74a-f5490731bd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.918677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252e655-317a-4ee2-a74a-f5490731bd1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.919516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252e655-317a-4ee2-a74a-f5490731bd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.919516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252e655-317a-4ee2-a74a-f5490731bd1e admin admin] PATH_INFO: `/v3/projects/b361133457c04e2da4c4158a45bbcf19/users/15932bb7b3044a41942b4e0edecebf10/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.921896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1139/2265] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:27:03.925932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe2141-38a7-4d01-834e-31051ec06a26 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:03.939365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252e655-317a-4ee2-a74a-f5490731bd1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b361133457c04e2da4c4158a45bbcf19, user_id=15932bb7b3044a41942b4e0edecebf10, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.940366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe2141-38a7-4d01-834e-31051ec06a26 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:03.943087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252e655-317a-4ee2-a74a-f5490731bd1e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:03.945327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2007e-01aa-41bc-bff1-65a7f9766216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:03.946358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2007e-01aa-41bc-bff1-65a7f9766216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:03.946648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2007e-01aa-41bc-bff1-65a7f9766216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:03.946948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2007e-01aa-41bc-bff1-65a7f9766216 admin admin] PATH_INFO: `/v3/projects/169e28d4b39c49b795dc95a74b664f07/users/f8495074b38d44ddabdaff69765ee378/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:03.962875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2007e-01aa-41bc-bff1-65a7f9766216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169e28d4b39c49b795dc95a74b664f07, user_id=f8495074b38d44ddabdaff69765ee378, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:03.966221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2007e-01aa-41bc-bff1-65a7f9766216 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:04.051186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1127/2266] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:27:03 2022] PUT /identity/v3/projects/b361133457c04e2da4c4158a45bbcf19/users/15932bb7b3044a41942b4e0edecebf10/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:27:04.056796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd190e-90d9-4927-8867-d5c72aa58fbf None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:04.056796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd190e-90d9-4927-8867-d5c72aa58fbf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:04.056796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd190e-90d9-4927-8867-d5c72aa58fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:04.079022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1140/2267] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:27:03 2022] PUT /identity/v3/projects/169e28d4b39c49b795dc95a74b664f07/users/f8495074b38d44ddabdaff69765ee378/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:27:04.089093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad122acf-fdf2-4edd-9063-400fe687bdb6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:04.089446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad122acf-fdf2-4edd-9063-400fe687bdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:04.089748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad122acf-fdf2-4edd-9063-400fe687bdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:04.090619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fabd190e-90d9-4927-8867-d5c72aa58fbf None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:04.091838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fabd190e-90d9-4927-8867-d5c72aa58fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:04.124733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ad122acf-fdf2-4edd-9063-400fe687bdb6 None None] MFA Rules not processed for user `f8495074b38d44ddabdaff69765ee378`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:04.125506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ad122acf-fdf2-4edd-9063-400fe687bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:04.153080 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1128/2268] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:04.163302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd190e-90d9-4927-8867-d5c72aa58fbf None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:04.164662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fabd190e-90d9-4927-8867-d5c72aa58fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:04.194144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1141/2269] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:04.204842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad122acf-fdf2-4edd-9063-400fe687bdb6 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:04.206352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ad122acf-fdf2-4edd-9063-400fe687bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:04.225820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd190e-90d9-4927-8867-d5c72aa58fbf None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:04.230351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777d5b9-b1e1-4b47-a7b1-9c66d80cfbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:04.231061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777d5b9-b1e1-4b47-a7b1-9c66d80cfbce service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:04.231346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777d5b9-b1e1-4b47-a7b1-9c66d80cfbce service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:04.231647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777d5b9-b1e1-4b47-a7b1-9c66d80cfbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:04.241061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777d5b9-b1e1-4b47-a7b1-9c66d80cfbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:04.242538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777d5b9-b1e1-4b47-a7b1-9c66d80cfbce service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:04.252322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1129/2270] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:27:04.265459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad122acf-fdf2-4edd-9063-400fe687bdb6 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:04.269241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:04.269917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:04.270207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:04.274234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:04.279051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:04.280536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:04.289403 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1142/2271] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:27:05.741473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f1520-dcfb-49a0-9d3a-db44a246c72b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:05.742035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f1520-dcfb-49a0-9d3a-db44a246c72b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:05.742188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f1520-dcfb-49a0-9d3a-db44a246c72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:05.770282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-dc9f1520-dcfb-49a0-9d3a-db44a246c72b None None] MFA Rules not processed for user `15932bb7b3044a41942b4e0edecebf10`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:05.771406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-dc9f1520-dcfb-49a0-9d3a-db44a246c72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:05.838683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1130/2272] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:05.849250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:05.850674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:05.866666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558393bc-df64-4501-a24b-d16f36abc6ec service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:05.867090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea69b3-fa35-4349-84f8-ac7712fda890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:05.867799 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea69b3-fa35-4349-84f8-ac7712fda890 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:05.868089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea69b3-fa35-4349-84f8-ac7712fda890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:05.868377 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea69b3-fa35-4349-84f8-ac7712fda890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:05.878062 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea69b3-fa35-4349-84f8-ac7712fda890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:05.879510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea69b3-fa35-4349-84f8-ac7712fda890 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:05.888426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1143/2273] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:27:09.275178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375e97d-d47d-43b4-942e-a8413278d9b8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:09.275644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375e97d-d47d-43b4-942e-a8413278d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:09.276033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375e97d-d47d-43b4-942e-a8413278d9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:09.293127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5375e97d-d47d-43b4-942e-a8413278d9b8 None None] MFA Rules not processed for user `f56c908ada31445e85ec1dfee8e2ebdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:09.294353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5375e97d-d47d-43b4-942e-a8413278d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:09.347094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1131/2274] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:11.072575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8e136-1d44-4320-81fa-7706fc582a1c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:11.073131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8e136-1d44-4320-81fa-7706fc582a1c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:11.073703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8e136-1d44-4320-81fa-7706fc582a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:11.120739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0ef8e136-1d44-4320-81fa-7706fc582a1c None None] MFA Rules not processed for user `15932bb7b3044a41942b4e0edecebf10`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:11.122224 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0ef8e136-1d44-4320-81fa-7706fc582a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:11.131385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1144/2275] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:11.487317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a348f6-0e2f-4e73-8217-1fca5c7d3f1c None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:11.488234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a348f6-0e2f-4e73-8217-1fca5c7d3f1c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:11.490411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a348f6-0e2f-4e73-8217-1fca5c7d3f1c None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:11.490411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1132/2276] 10.222.0.25 () {54 vars in 912 bytes} [Fri May 13 13:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:27:11.496337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493127-c135-425e-aa51-c6e065f9f407 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:11.497398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493127-c135-425e-aa51-c6e065f9f407 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:11.497752 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493127-c135-425e-aa51-c6e065f9f407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:11.539788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b4493127-c135-425e-aa51-c6e065f9f407 None None] MFA Rules not processed for user `8ddb0dcaa5304016af1bc1f88845fb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:11.541255 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b4493127-c135-425e-aa51-c6e065f9f407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:11.550150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1145/2277] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:27:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:11.563375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a348f6-0e2f-4e73-8217-1fca5c7d3f1c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:11.591061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a348f6-0e2f-4e73-8217-1fca5c7d3f1c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:11.598668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc21474-51d3-4f53-80fe-cb7a4433cf00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:11.599455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21474-51d3-4f53-80fe-cb7a4433cf00 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:11.599595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21474-51d3-4f53-80fe-cb7a4433cf00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:11.599865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21474-51d3-4f53-80fe-cb7a4433cf00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:11.608312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc21474-51d3-4f53-80fe-cb7a4433cf00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:11.610040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc21474-51d3-4f53-80fe-cb7a4433cf00 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:11.619312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1133/2278] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:27:15.109147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493127-c135-425e-aa51-c6e065f9f407 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:15.120698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493127-c135-425e-aa51-c6e065f9f407 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:15.125777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:15.126672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:15.127070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:15.127359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] PATH_INFO: `/v3/users/f8495074b38d44ddabdaff69765ee378` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:15.128559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8495074b38d44ddabdaff69765ee378)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:15.130539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:15.299321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8495074b38d44ddabdaff69765ee378'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:15.300434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8495074b38d44ddabdaff69765ee378'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:15.428625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8495074b38d44ddabdaff69765ee378'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:15.437602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8495074b38d44ddabdaff69765ee378'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:15.440494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8495074b38d44ddabdaff69765ee378'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:15.469609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1146/2279] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:27:15 2022] DELETE /identity/v3/users/f8495074b38d44ddabdaff69765ee378 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:15.474344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249cf00-0840-4b7e-a2b5-de7384dcce4b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:15.474874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249cf00-0840-4b7e-a2b5-de7384dcce4b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:15.474960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249cf00-0840-4b7e-a2b5-de7384dcce4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:15.494315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5249cf00-0840-4b7e-a2b5-de7384dcce4b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:15.495554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5249cf00-0840-4b7e-a2b5-de7384dcce4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:15.566711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1134/2280] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:15.573746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:15.584185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:15.657147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b6d7c-27b3-424e-bbec-d3414cade59b admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:15.661431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1c375-763f-4fae-9c25-7bf7c524aad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:15.662384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1c375-763f-4fae-9c25-7bf7c524aad3 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:15.663267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1c375-763f-4fae-9c25-7bf7c524aad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:15.663675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1c375-763f-4fae-9c25-7bf7c524aad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:15.674672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1c375-763f-4fae-9c25-7bf7c524aad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:15.676286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1c375-763f-4fae-9c25-7bf7c524aad3 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:15.687184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1147/2281] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:27:16.379215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249cf00-0840-4b7e-a2b5-de7384dcce4b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:16.380702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5249cf00-0840-4b7e-a2b5-de7384dcce4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:16.392686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249cf00-0840-4b7e-a2b5-de7384dcce4b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:16.397325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:16.398343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:16.410894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:16.411823 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] PATH_INFO: `/v3/projects/169e28d4b39c49b795dc95a74b664f07` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:16.414153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=169e28d4b39c49b795dc95a74b664f07)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:16.416102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:16.565128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa437aca-4536-4b64-b7a7-afc5c5721cc1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:16.565128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa437aca-4536-4b64-b7a7-afc5c5721cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:16.565128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa437aca-4536-4b64-b7a7-afc5c5721cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:16.580544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-aa437aca-4536-4b64-b7a7-afc5c5721cc1 None None] MFA Rules not processed for user `1cdee89f8a8040ca944458d3fc6f18eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:16.582003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-aa437aca-4536-4b64-b7a7-afc5c5721cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:16.605175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169e28d4b39c49b795dc95a74b664f07', 'request_id': 'req-3944f980-adc4-46cf-aa0d-14b5e158e74a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:16.614058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169e28d4b39c49b795dc95a74b664f07', 'request_id': 'req-3944f980-adc4-46cf-aa0d-14b5e158e74a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:16.653948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1148/2282] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:16.719558 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169e28d4b39c49b795dc95a74b664f07', 'request_id': 'req-3944f980-adc4-46cf-aa0d-14b5e158e74a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:16.743104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] The token cache is being invalidate because project 169e28d4b39c49b795dc95a74b664f07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:27:16.743472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:16.745751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3944f980-adc4-46cf-aa0d-14b5e158e74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:16.749833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1135/2283] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:27:16 2022] DELETE /identity/v3/projects/169e28d4b39c49b795dc95a74b664f07 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:16.753568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcb05f-5c20-4484-abb0-b6eb1231813b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:16.756979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcb05f-5c20-4484-abb0-b6eb1231813b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:16.756979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcb05f-5c20-4484-abb0-b6eb1231813b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:16.774072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-30fcb05f-5c20-4484-abb0-b6eb1231813b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:16.776076 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-30fcb05f-5c20-4484-abb0-b6eb1231813b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:16.849815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1149/2284] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:16.859734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057679cd-06ad-44e5-aa99-34cc522545fa None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:16.860296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057679cd-06ad-44e5-aa99-34cc522545fa None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:16.860765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057679cd-06ad-44e5-aa99-34cc522545fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:16.881739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-057679cd-06ad-44e5-aa99-34cc522545fa None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:16.882982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-057679cd-06ad-44e5-aa99-34cc522545fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:16.893286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1136/2285] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:16.900313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcb05f-5c20-4484-abb0-b6eb1231813b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:16.918347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcb05f-5c20-4484-abb0-b6eb1231813b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:16.927660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:16.928978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:16.933445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:16.933445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:16.933445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:27:16.933445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:16.933445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:16.952141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1150/2286] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:27:16.958323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057679cd-06ad-44e5-aa99-34cc522545fa None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:16.968732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057679cd-06ad-44e5-aa99-34cc522545fa None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:16.974699 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fef6b-da7e-48e6-a5c5-acb766657377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:16.975455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fef6b-da7e-48e6-a5c5-acb766657377 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:16.975737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fef6b-da7e-48e6-a5c5-acb766657377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:16.976013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fef6b-da7e-48e6-a5c5-acb766657377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:16.976710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fef6b-da7e-48e6-a5c5-acb766657377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:16.978678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fef6b-da7e-48e6-a5c5-acb766657377 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:17.127499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1137/2287] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:27:16 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:27:17.128816 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:17.143251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42932-a31a-4d0f-9495-d43a1b63b252 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:17.223455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faef243-121f-4f52-80fd-61b1f21b3970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:17.223676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faef243-121f-4f52-80fd-61b1f21b3970 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:17.223907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faef243-121f-4f52-80fd-61b1f21b3970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:17.224187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faef243-121f-4f52-80fd-61b1f21b3970 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:17.224889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faef243-121f-4f52-80fd-61b1f21b3970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:17.226908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faef243-121f-4f52-80fd-61b1f21b3970 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:17.293176 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1151/2288] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:27:17 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:27:17.296821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fef6b-da7e-48e6-a5c5-acb766657377 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:17.309164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fef6b-da7e-48e6-a5c5-acb766657377 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:17.315085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607db59-fd54-4856-8fc1-e638da35b880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:17.315842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607db59-fd54-4856-8fc1-e638da35b880 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:17.316208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607db59-fd54-4856-8fc1-e638da35b880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:17.316576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607db59-fd54-4856-8fc1-e638da35b880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:17.317517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607db59-fd54-4856-8fc1-e638da35b880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:17.319872 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607db59-fd54-4856-8fc1-e638da35b880 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:17.335828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1138/2289] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:27:17.349729 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faef243-121f-4f52-80fd-61b1f21b3970 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:17.357456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faef243-121f-4f52-80fd-61b1f21b3970 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:17.363762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50a4dd-174f-4425-9689-e7f0552d9929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:17.364517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50a4dd-174f-4425-9689-e7f0552d9929 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:17.364884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50a4dd-174f-4425-9689-e7f0552d9929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:17.365218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50a4dd-174f-4425-9689-e7f0552d9929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:17.366328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50a4dd-174f-4425-9689-e7f0552d9929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:17.368032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50a4dd-174f-4425-9689-e7f0552d9929 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:17.384126 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1152/2290] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:27:17.389451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607db59-fd54-4856-8fc1-e638da35b880 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:17.406142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607db59-fd54-4856-8fc1-e638da35b880 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:17.411584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:17.412626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:17.413006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:17.413358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] PATH_INFO: `/v3/projects/7063f805431a433bab3249108e0443af/users/33ac4c1b17484e24a54e95b3fcfc1031/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:17.431714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7063f805431a433bab3249108e0443af, user_id=33ac4c1b17484e24a54e95b3fcfc1031, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:17.434893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:17.585068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1139/2291] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:27:17 2022] PUT /identity/v3/projects/7063f805431a433bab3249108e0443af/users/33ac4c1b17484e24a54e95b3fcfc1031/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:17.588612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8f06-1c0c-46ac-a34c-8a7722c788ff None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:17.589068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8f06-1c0c-46ac-a34c-8a7722c788ff None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:17.590409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c8f06-1c0c-46ac-a34c-8a7722c788ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:17.631893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-a25c8f06-1c0c-46ac-a34c-8a7722c788ff None None] MFA Rules not processed for user `33ac4c1b17484e24a54e95b3fcfc1031`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:17.633741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a25c8f06-1c0c-46ac-a34c-8a7722c788ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:17.732373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1153/2292] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:17.743079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:17.751167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:17.862876 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a31707-6004-41a8-88a2-829f5e696026 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:17.866589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf870ea-6310-48c8-961f-c5ba874c107b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3f786b791b4c8796d251948917ca93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:17.867233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf870ea-6310-48c8-961f-c5ba874c107b service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:17.867516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf870ea-6310-48c8-961f-c5ba874c107b service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:17.867795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf870ea-6310-48c8-961f-c5ba874c107b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:17.876744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf870ea-6310-48c8-961f-c5ba874c107b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:17.878315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf870ea-6310-48c8-961f-c5ba874c107b service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:17.887115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1140/2293] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:17 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:27:20.976060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c8f06-1c0c-46ac-a34c-8a7722c788ff None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:20.977977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a25c8f06-1c0c-46ac-a34c-8a7722c788ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:21.096771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c8f06-1c0c-46ac-a34c-8a7722c788ff None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:21.103277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:21.105192 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:21.105675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:21.106209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] PATH_INFO: `/v3/users/5e83e2f7adfb4f77aabfb3ddae66a3ab` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:21.109089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e83e2f7adfb4f77aabfb3ddae66a3ab)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:21.111523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:21.309406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e83e2f7adfb4f77aabfb3ddae66a3ab'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:21.312072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e83e2f7adfb4f77aabfb3ddae66a3ab'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:21.434029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e83e2f7adfb4f77aabfb3ddae66a3ab'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:21.443600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e83e2f7adfb4f77aabfb3ddae66a3ab'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:21.444382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e83e2f7adfb4f77aabfb3ddae66a3ab'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:21.462923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1154/2294] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:27:20 2022] DELETE /identity/v3/users/5e83e2f7adfb4f77aabfb3ddae66a3ab => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:21.467528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eaf04-276d-4a38-b940-22c7fd4df873 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:21.468004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eaf04-276d-4a38-b940-22c7fd4df873 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:21.468177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eaf04-276d-4a38-b940-22c7fd4df873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:21.486587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f57eaf04-276d-4a38-b940-22c7fd4df873 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:21.487904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f57eaf04-276d-4a38-b940-22c7fd4df873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:21.559299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1141/2295] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:21.569566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:21.570994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:21.630212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733493e-5bfd-451c-8782-82bf3ec6895e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:21.634112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359788d0-88a5-4a64-b3e1-e33955a1602a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:21.635566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359788d0-88a5-4a64-b3e1-e33955a1602a service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:21.635566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359788d0-88a5-4a64-b3e1-e33955a1602a service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:21.635566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359788d0-88a5-4a64-b3e1-e33955a1602a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:21.645246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359788d0-88a5-4a64-b3e1-e33955a1602a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:21.647240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359788d0-88a5-4a64-b3e1-e33955a1602a service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:21.658660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1155/2296] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:27:22.263408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57eaf04-276d-4a38-b940-22c7fd4df873 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:22.266494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f57eaf04-276d-4a38-b940-22c7fd4df873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:22.279219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57eaf04-276d-4a38-b940-22c7fd4df873 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:22.284990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:22.286348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:22.286784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:22.287223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] PATH_INFO: `/v3/projects/a01bc388c3fb4582a5fd6107058310c0` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:22.288675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a01bc388c3fb4582a5fd6107058310c0)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:22.291434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:22.430181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a01bc388c3fb4582a5fd6107058310c0', 'request_id': 'req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:22.437294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a01bc388c3fb4582a5fd6107058310c0', 'request_id': 'req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:22.502441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a01bc388c3fb4582a5fd6107058310c0', 'request_id': 'req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:22.515324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] The token cache is being invalidate because project a01bc388c3fb4582a5fd6107058310c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:27:22.515741 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:22.516548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1d384ed1-0ccd-4d71-8c73-cc6f1c9cf42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:22.518867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1142/2297] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:27:22 2022] DELETE /identity/v3/projects/a01bc388c3fb4582a5fd6107058310c0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:22.524228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177ecf-308f-49c9-8b9c-551fa4a9ba99 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:22.525680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177ecf-308f-49c9-8b9c-551fa4a9ba99 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:22.525680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e177ecf-308f-49c9-8b9c-551fa4a9ba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:22.542637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0e177ecf-308f-49c9-8b9c-551fa4a9ba99 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:22.543810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0e177ecf-308f-49c9-8b9c-551fa4a9ba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:22.604050 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1156/2298] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:22.611880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8860c8-7076-4651-b61f-225e830bc5df None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:22.612173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8860c8-7076-4651-b61f-225e830bc5df None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:22.612461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8860c8-7076-4651-b61f-225e830bc5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:22.631344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3d8860c8-7076-4651-b61f-225e830bc5df None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:22.632544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3d8860c8-7076-4651-b61f-225e830bc5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:22.641129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1143/2299] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:22.645375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e177ecf-308f-49c9-8b9c-551fa4a9ba99 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:22.655376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e177ecf-308f-49c9-8b9c-551fa4a9ba99 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:22.659978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:22.660645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:22.660928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:22.661203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:22.661978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:27:22.662282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:22.663896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:22.677929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1157/2300] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:27:22.683295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8860c8-7076-4651-b61f-225e830bc5df None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:22.693780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8860c8-7076-4651-b61f-225e830bc5df None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:22.698443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a428-059a-49e9-9540-1553db127348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:22.699344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a428-059a-49e9-9540-1553db127348 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:22.699652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a428-059a-49e9-9540-1553db127348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:22.699955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a428-059a-49e9-9540-1553db127348 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:22.700643 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a428-059a-49e9-9540-1553db127348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:22.703234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a428-059a-49e9-9540-1553db127348 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:22.782610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1144/2301] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:27:22 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:27:22.788167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:22.799867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4428c88-eb94-441f-9ff5-f7ea695ab465 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:22.862121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f74112-34cf-40d5-b91f-6f757d6695de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:22.862807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f74112-34cf-40d5-b91f-6f757d6695de admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:22.863089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f74112-34cf-40d5-b91f-6f757d6695de admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:22.863360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f74112-34cf-40d5-b91f-6f757d6695de admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:22.864044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f74112-34cf-40d5-b91f-6f757d6695de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:22.866205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f74112-34cf-40d5-b91f-6f757d6695de admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:22.980834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1158/2302] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:27:22 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:27:22.987600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a428-059a-49e9-9540-1553db127348 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:22.998676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a428-059a-49e9-9540-1553db127348 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:23.004149 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4486eb-714f-41c8-af42-94ec8946a840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:23.005695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4486eb-714f-41c8-af42-94ec8946a840 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:23.005695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4486eb-714f-41c8-af42-94ec8946a840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:23.005695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4486eb-714f-41c8-af42-94ec8946a840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:23.008014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4486eb-714f-41c8-af42-94ec8946a840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:23.008302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4486eb-714f-41c8-af42-94ec8946a840 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:23.024430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1145/2303] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:27:23.025688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f74112-34cf-40d5-b91f-6f757d6695de admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:23.039506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f74112-34cf-40d5-b91f-6f757d6695de admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:23.049993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9eff5-068f-41c8-9d9d-f125a31af140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:23.051683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9eff5-068f-41c8-9d9d-f125a31af140 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:23.052469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9eff5-068f-41c8-9d9d-f125a31af140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:23.053612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9eff5-068f-41c8-9d9d-f125a31af140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:23.054681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9eff5-068f-41c8-9d9d-f125a31af140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:23.065476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9eff5-068f-41c8-9d9d-f125a31af140 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:23.080746 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1159/2304] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:27:23.084759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4486eb-714f-41c8-af42-94ec8946a840 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:23.106216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4486eb-714f-41c8-af42-94ec8946a840 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:23.124546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:23.126375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:23.126746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:23.127100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] PATH_INFO: `/v3/projects/a0b052b05d124da69054f7bbc2f1a986/users/d0e60d53623a47ab83b84c9627073bef/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:23.152649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b052b05d124da69054f7bbc2f1a986, user_id=d0e60d53623a47ab83b84c9627073bef, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:23.156312 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:23.226966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1146/2305] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:27:23 2022] PUT /identity/v3/projects/a0b052b05d124da69054f7bbc2f1a986/users/d0e60d53623a47ab83b84c9627073bef/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:23.238213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6750b-ccdc-4b41-a7a7-0ba238beaf51 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:23.238514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6750b-ccdc-4b41-a7a7-0ba238beaf51 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:23.238920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6750b-ccdc-4b41-a7a7-0ba238beaf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:23.281491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-c7e6750b-ccdc-4b41-a7a7-0ba238beaf51 None None] MFA Rules not processed for user `d0e60d53623a47ab83b84c9627073bef`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:23.282759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c7e6750b-ccdc-4b41-a7a7-0ba238beaf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:23.348728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1160/2306] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:23.361248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:23.362984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:23.422451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd41aa1-a1e9-4c19-a997-88c23eaa16e8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:23.430057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:23.430057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:23.430057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:23.430057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:23.442933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:23.444456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:23.455089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1147/2307] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:27:24.402478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf5ea-6b26-438a-b9a9-79fe0aab757e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:24.403465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf5ea-6b26-438a-b9a9-79fe0aab757e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:24.403948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf5ea-6b26-438a-b9a9-79fe0aab757e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:24.442032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-84fbf5ea-6b26-438a-b9a9-79fe0aab757e None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:24.447402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-84fbf5ea-6b26-438a-b9a9-79fe0aab757e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:24.589119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1161/2308] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:27:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:24.632508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:24.642396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:24.726204 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080444-b5a9-4b59-9fda-b3c8da842bbe service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:24.731895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45d573-4247-4b5f-af3a-d83554f5ae3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:24.740049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45d573-4247-4b5f-af3a-d83554f5ae3a service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:24.740049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45d573-4247-4b5f-af3a-d83554f5ae3a service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:24.740049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45d573-4247-4b5f-af3a-d83554f5ae3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:24.742966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45d573-4247-4b5f-af3a-d83554f5ae3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:24.744778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45d573-4247-4b5f-af3a-d83554f5ae3a service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:24.753974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1148/2309] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:27:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:27:48.762489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e5bf-03a8-476c-8104-dedd731484e9 None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:48.763042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e5bf-03a8-476c-8104-dedd731484e9 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:48.763179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e5bf-03a8-476c-8104-dedd731484e9 None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:48.764748 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1162/2310] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:27:48.770648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2dc7f-ec4a-4acf-b2b8-4cd9371b2dd3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:48.771158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2dc7f-ec4a-4acf-b2b8-4cd9371b2dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:48.771285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2dc7f-ec4a-4acf-b2b8-4cd9371b2dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:48.821393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c2a2dc7f-ec4a-4acf-b2b8-4cd9371b2dd3 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:48.822787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c2a2dc7f-ec4a-4acf-b2b8-4cd9371b2dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:48.909015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1149/2311] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:27:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:48.918133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252e5bf-03a8-476c-8104-dedd731484e9 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:48.919836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f252e5bf-03a8-476c-8104-dedd731484e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:48.998995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252e5bf-03a8-476c-8104-dedd731484e9 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:49.004162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d424d-fe49-47e0-9199-e9e6dd6f170c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3f786b791b4c8796d251948917ca93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:49.004822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d424d-fe49-47e0-9199-e9e6dd6f170c service glance] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:49.005106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d424d-fe49-47e0-9199-e9e6dd6f170c service glance] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:49.005383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d424d-fe49-47e0-9199-e9e6dd6f170c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:49.016154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d424d-fe49-47e0-9199-e9e6dd6f170c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:49.017615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d424d-fe49-47e0-9199-e9e6dd6f170c service glance] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:49.028166 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1163/2312] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:27:49.256902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b717ae-dc8d-4bf2-8b99-99b9c4021f89 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:49.257110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b717ae-dc8d-4bf2-8b99-99b9c4021f89 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:49.257371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b717ae-dc8d-4bf2-8b99-99b9c4021f89 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:49.258834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1150/2313] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:27:49.263688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31790c4c-9d47-4367-ad00-b2a2db1c0ce6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:49.263981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31790c4c-9d47-4367-ad00-b2a2db1c0ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:49.264285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31790c4c-9d47-4367-ad00-b2a2db1c0ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:49.281852 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-31790c4c-9d47-4367-ad00-b2a2db1c0ce6 None None] MFA Rules not processed for user `8ddb0dcaa5304016af1bc1f88845fb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:49.283084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-31790c4c-9d47-4367-ad00-b2a2db1c0ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:49.344010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1164/2314] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:27:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:49.350026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b717ae-dc8d-4bf2-8b99-99b9c4021f89 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:49.351838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-30b717ae-dc8d-4bf2-8b99-99b9c4021f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:49.365422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b717ae-dc8d-4bf2-8b99-99b9c4021f89 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:49.368938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8276e-aeca-481f-8e0d-16ce3cb502ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:49.369662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8276e-aeca-481f-8e0d-16ce3cb502ab service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:49.369980 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8276e-aeca-481f-8e0d-16ce3cb502ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:49.370257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8276e-aeca-481f-8e0d-16ce3cb502ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:49.378093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8276e-aeca-481f-8e0d-16ce3cb502ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:49.379555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8276e-aeca-481f-8e0d-16ce3cb502ab service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:49.387373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1151/2315] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:27:49.475911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868ceba-d615-4852-bcd7-b493a12b3c2f None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:49.475911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868ceba-d615-4852-bcd7-b493a12b3c2f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:49.475911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868ceba-d615-4852-bcd7-b493a12b3c2f None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:49.475911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1165/2316] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:27:49.478877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401be99-cc71-4f2a-bc1e-0a4abc25321e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:49.479151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401be99-cc71-4f2a-bc1e-0a4abc25321e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:49.479432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401be99-cc71-4f2a-bc1e-0a4abc25321e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:49.497909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d401be99-cc71-4f2a-bc1e-0a4abc25321e None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:49.499762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d401be99-cc71-4f2a-bc1e-0a4abc25321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:49.509346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1152/2317] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:27:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:49.519760 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868ceba-d615-4852-bcd7-b493a12b3c2f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:49.531027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868ceba-d615-4852-bcd7-b493a12b3c2f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:49.534884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ae710-8548-40b8-a3aa-0d16f7eb8005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:49.535539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae710-8548-40b8-a3aa-0d16f7eb8005 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:49.535826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae710-8548-40b8-a3aa-0d16f7eb8005 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:49.536136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae710-8548-40b8-a3aa-0d16f7eb8005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:49.548031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ae710-8548-40b8-a3aa-0d16f7eb8005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:49.549547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ae710-8548-40b8-a3aa-0d16f7eb8005 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:49.561679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1166/2318] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:27:54.683552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d81f4e-575b-4866-a3c4-e23a7915c61c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:54.684058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d81f4e-575b-4866-a3c4-e23a7915c61c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:54.684170 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d81f4e-575b-4866-a3c4-e23a7915c61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:54.733443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d0d81f4e-575b-4866-a3c4-e23a7915c61c None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:54.734676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d0d81f4e-575b-4866-a3c4-e23a7915c61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:54.751490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1153/2319] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:27:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:54.764346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ae710-8548-40b8-a3aa-0d16f7eb8005 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:54.786573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ae710-8548-40b8-a3aa-0d16f7eb8005 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:54.794349 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce3a0-04ec-476f-81e0-13ea5a89456c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:54.795026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce3a0-04ec-476f-81e0-13ea5a89456c service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:54.795330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce3a0-04ec-476f-81e0-13ea5a89456c service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:54.795590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce3a0-04ec-476f-81e0-13ea5a89456c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:54.804044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce3a0-04ec-476f-81e0-13ea5a89456c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:54.805681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce3a0-04ec-476f-81e0-13ea5a89456c service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:54.813623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1167/2320] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:27:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:27:55.378305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d81f4e-575b-4866-a3c4-e23a7915c61c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:55.438914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d81f4e-575b-4866-a3c4-e23a7915c61c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:55.443486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:55.444347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:55.444634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:55.444908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] PATH_INFO: `/v3/users/d0e60d53623a47ab83b84c9627073bef` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:55.446138 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e60d53623a47ab83b84c9627073bef)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:55.448102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:55.647078 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e60d53623a47ab83b84c9627073bef'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:55.647737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e60d53623a47ab83b84c9627073bef'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:55.648338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e60d53623a47ab83b84c9627073bef'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:55.655465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e60d53623a47ab83b84c9627073bef'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:55.742662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e60d53623a47ab83b84c9627073bef'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:55.761672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1154/2321] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:27:55 2022] DELETE /identity/v3/users/d0e60d53623a47ab83b84c9627073bef => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:55.766421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa3896-b506-42b0-88ac-7a2c22eb1f54 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:55.766927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa3896-b506-42b0-88ac-7a2c22eb1f54 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:55.767024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa3896-b506-42b0-88ac-7a2c22eb1f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:55.784693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1aaa3896-b506-42b0-88ac-7a2c22eb1f54 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:55.785933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1aaa3896-b506-42b0-88ac-7a2c22eb1f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:55.857015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1168/2322] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:55.864946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:55.871828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:55.984317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cd0d2-ec95-48fd-b884-c6f80b9b582c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:55.988557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e69f6-85e0-495e-9174-b889e49ea1ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:55.989296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e69f6-85e0-495e-9174-b889e49ea1ea service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:55.989641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e69f6-85e0-495e-9174-b889e49ea1ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:55.990366 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e69f6-85e0-495e-9174-b889e49ea1ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:56.002002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e69f6-85e0-495e-9174-b889e49ea1ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:56.003693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e69f6-85e0-495e-9174-b889e49ea1ea service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:56.022836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1155/2323] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:27:56.908862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa3896-b506-42b0-88ac-7a2c22eb1f54 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:56.910323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1aaa3896-b506-42b0-88ac-7a2c22eb1f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:56.923195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa3896-b506-42b0-88ac-7a2c22eb1f54 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:56.929211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:56.930390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:56.931318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:56.931318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] PATH_INFO: `/v3/projects/a0b052b05d124da69054f7bbc2f1a986` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:56.932355 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b052b05d124da69054f7bbc2f1a986)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:56.934391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:57.074430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b052b05d124da69054f7bbc2f1a986', 'request_id': 'req-7647b183-0291-4d89-b039-1ad2ce6c35b7'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:57.080484 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b052b05d124da69054f7bbc2f1a986', 'request_id': 'req-7647b183-0291-4d89-b039-1ad2ce6c35b7'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:57.169180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b052b05d124da69054f7bbc2f1a986', 'request_id': 'req-7647b183-0291-4d89-b039-1ad2ce6c35b7'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:57.184098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] The token cache is being invalidate because project a0b052b05d124da69054f7bbc2f1a986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:27:57.184712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:57.185539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7647b183-0291-4d89-b039-1ad2ce6c35b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:27:57.197819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1169/2324] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:27:56 2022] DELETE /identity/v3/projects/a0b052b05d124da69054f7bbc2f1a986 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:57.203287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973382-56e7-47d1-b06c-70896347ee00 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:57.203775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973382-56e7-47d1-b06c-70896347ee00 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:57.203877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973382-56e7-47d1-b06c-70896347ee00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:57.232290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fc973382-56e7-47d1-b06c-70896347ee00 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:57.234155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fc973382-56e7-47d1-b06c-70896347ee00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:57.306582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1156/2325] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:27:57.314384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90125c7c-934c-4f03-9386-6048016526aa None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:57.314679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90125c7c-934c-4f03-9386-6048016526aa None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:57.314957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90125c7c-934c-4f03-9386-6048016526aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:57.333213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-90125c7c-934c-4f03-9386-6048016526aa None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:57.334452 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-90125c7c-934c-4f03-9386-6048016526aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:57.342937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1170/2326] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:57.347237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973382-56e7-47d1-b06c-70896347ee00 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:57.360667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973382-56e7-47d1-b06c-70896347ee00 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:57.365462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:57.366315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:57.366604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:57.366890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:57.367628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:27:57.367939 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:57.369557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:57.381504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1157/2327] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:27:57.385935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90125c7c-934c-4f03-9386-6048016526aa None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:57.394980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90125c7c-934c-4f03-9386-6048016526aa None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:57.399608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbfc2c-74a1-45b1-a457-b912b7563a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:57.400249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbfc2c-74a1-45b1-a457-b912b7563a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:57.400586 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbfc2c-74a1-45b1-a457-b912b7563a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:57.400860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbfc2c-74a1-45b1-a457-b912b7563a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:57.401608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbfc2c-74a1-45b1-a457-b912b7563a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:57.403597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbfc2c-74a1-45b1-a457-b912b7563a99 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:57.505132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1171/2328] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:27:57 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:27:57.508873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:57.520679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e1f6e-3bbc-4719-85fd-a614bbbf6874 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:57.591977 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04c71-342a-4243-a90e-bd4bb647d245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:57.593105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04c71-342a-4243-a90e-bd4bb647d245 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:57.593396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04c71-342a-4243-a90e-bd4bb647d245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:57.597703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee04c71-342a-4243-a90e-bd4bb647d245 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:57.597703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee04c71-342a-4243-a90e-bd4bb647d245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:57.597703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee04c71-342a-4243-a90e-bd4bb647d245 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:58.128055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1158/2329] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:27:57 2022] POST /identity/v3/users => generated 293 bytes in 616 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:27:58.129372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbfc2c-74a1-45b1-a457-b912b7563a99 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:58.140549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbfc2c-74a1-45b1-a457-b912b7563a99 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:58.145334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a72db-d4f2-45f6-9bf1-486ff04e5e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:58.145946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a72db-d4f2-45f6-9bf1-486ff04e5e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:58.146230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a72db-d4f2-45f6-9bf1-486ff04e5e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:58.146512 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a72db-d4f2-45f6-9bf1-486ff04e5e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:58.147244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a72db-d4f2-45f6-9bf1-486ff04e5e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:58.148939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a72db-d4f2-45f6-9bf1-486ff04e5e94 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:58.160699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1172/2330] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:27:58.165490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04c71-342a-4243-a90e-bd4bb647d245 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:58.176049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee04c71-342a-4243-a90e-bd4bb647d245 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:58.188606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ebfeb-acb5-42a6-a042-2894b589fb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:58.189288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ebfeb-acb5-42a6-a042-2894b589fb23 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:58.189576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ebfeb-acb5-42a6-a042-2894b589fb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:58.189900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ebfeb-acb5-42a6-a042-2894b589fb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:58.190677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ebfeb-acb5-42a6-a042-2894b589fb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:58.192296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ebfeb-acb5-42a6-a042-2894b589fb23 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:58.204249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1159/2331] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:27:58.208693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a72db-d4f2-45f6-9bf1-486ff04e5e94 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:58.219283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a72db-d4f2-45f6-9bf1-486ff04e5e94 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:58.223974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:58.224940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:58.225328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:58.225506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] PATH_INFO: `/v3/projects/1503b2654ff7475b98032519919ae481/users/015fce3883ab4ee0960565c1a51f520d/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:58.242983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1503b2654ff7475b98032519919ae481, user_id=015fce3883ab4ee0960565c1a51f520d, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:58.245892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:58.323005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1173/2332] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:27:58 2022] PUT /identity/v3/projects/1503b2654ff7475b98032519919ae481/users/015fce3883ab4ee0960565c1a51f520d/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:27:58.329550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23830a9-0cea-437c-9f11-de6a5ba8ea10 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:58.329981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23830a9-0cea-437c-9f11-de6a5ba8ea10 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:58.330339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23830a9-0cea-437c-9f11-de6a5ba8ea10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:58.347642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d23830a9-0cea-437c-9f11-de6a5ba8ea10 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:27:58.348930 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d23830a9-0cea-437c-9f11-de6a5ba8ea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:58.414746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1160/2333] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:27:58.425124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:27:58.426876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:27:58.512651 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dd19c-cc5b-4f76-8435-f142fa938622 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:27:58.516502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:27:58.517475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:27:58.517475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:27:58.517712 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:27:58.528365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:27:58.530092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:27:58.544137 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1174/2334] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:28:03.160610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7586e-9781-44ed-8fc2-daaa0f2148e2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:03.161197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7586e-9781-44ed-8fc2-daaa0f2148e2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:03.161197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7586e-9781-44ed-8fc2-daaa0f2148e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:03.188079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-68f7586e-9781-44ed-8fc2-daaa0f2148e2 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:03.189553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-68f7586e-9781-44ed-8fc2-daaa0f2148e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:03.261305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1161/2335] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:28:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:03.274360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:03.276077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:03.380677 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d4afe-ddbb-4a55-b99f-5f22d352c4a4 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:03.395171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b4ef9-b3da-4603-a2e5-560b01294a79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:03.396381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4ef9-b3da-4603-a2e5-560b01294a79 service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:03.396966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4ef9-b3da-4603-a2e5-560b01294a79 service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:03.397707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4ef9-b3da-4603-a2e5-560b01294a79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:03.409139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b4ef9-b3da-4603-a2e5-560b01294a79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:03.411899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b4ef9-b3da-4603-a2e5-560b01294a79 service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:03.425106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1175/2336] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:28:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 13 13:28:07.360974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221719c-3180-49c0-9491-e6c3df9e4c3d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:07.362137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221719c-3180-49c0-9491-e6c3df9e4c3d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:07.362425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221719c-3180-49c0-9491-e6c3df9e4c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:07.411337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6221719c-3180-49c0-9491-e6c3df9e4c3d None None] MFA Rules not processed for user `015fce3883ab4ee0960565c1a51f520d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:07.413178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6221719c-3180-49c0-9491-e6c3df9e4c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:07.581320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1162/2337] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:28:07.582900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b4ef9-b3da-4603-a2e5-560b01294a79 service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:07.594812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b4ef9-b3da-4603-a2e5-560b01294a79 service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:07.601941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d406d-23ec-4ff1-8434-a53cd0f82a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:07.601941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d406d-23ec-4ff1-8434-a53cd0f82a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:07.601941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d406d-23ec-4ff1-8434-a53cd0f82a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:07.601941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d406d-23ec-4ff1-8434-a53cd0f82a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:07.602490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d406d-23ec-4ff1-8434-a53cd0f82a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:07.604758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d406d-23ec-4ff1-8434-a53cd0f82a8a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:07.719947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1176/2338] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:28:07 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:28:07.720624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221719c-3180-49c0-9491-e6c3df9e4c3d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:07.740675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221719c-3180-49c0-9491-e6c3df9e4c3d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:07.906050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bc46d-3b89-497b-a04e-a60a85ce76ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:07.907138 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc46d-3b89-497b-a04e-a60a85ce76ca admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:07.907618 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc46d-3b89-497b-a04e-a60a85ce76ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:07.908484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc46d-3b89-497b-a04e-a60a85ce76ca admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:07.909560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bc46d-3b89-497b-a04e-a60a85ce76ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:07.912962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bc46d-3b89-497b-a04e-a60a85ce76ca admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:08.016568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1163/2339] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:28:07 2022] POST /identity/v3/users => generated 299 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:28:08.021174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d406d-23ec-4ff1-8434-a53cd0f82a8a admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:08.052135 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d406d-23ec-4ff1-8434-a53cd0f82a8a admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:08.073762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e93fa9-99cd-4df4-81be-8c1c46e28cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:08.074668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e93fa9-99cd-4df4-81be-8c1c46e28cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:08.075095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e93fa9-99cd-4df4-81be-8c1c46e28cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:08.075833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e93fa9-99cd-4df4-81be-8c1c46e28cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:08.077290 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e93fa9-99cd-4df4-81be-8c1c46e28cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:08.089197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e93fa9-99cd-4df4-81be-8c1c46e28cdf admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:08.124466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1177/2340] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:28:08.125732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bc46d-3b89-497b-a04e-a60a85ce76ca admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:08.157138 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bc46d-3b89-497b-a04e-a60a85ce76ca admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:08.180144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063454-ace0-4917-b44f-1c04380efced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:08.186003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063454-ace0-4917-b44f-1c04380efced admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:08.186361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063454-ace0-4917-b44f-1c04380efced admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:08.186361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063454-ace0-4917-b44f-1c04380efced admin admin] PATH_INFO: `/v3/projects/79274e9a4e764aa8999b9ee551b0bc6e/users/afbdf2e06ca249b0b037b28750c444a8/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:08.249518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26063454-ace0-4917-b44f-1c04380efced admin admin] RBAC: Authorizing `identity:create_grant(project_id=79274e9a4e764aa8999b9ee551b0bc6e, user_id=afbdf2e06ca249b0b037b28750c444a8, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:08.254990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26063454-ace0-4917-b44f-1c04380efced admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:08.372600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1164/2341] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:28:08 2022] PUT /identity/v3/projects/79274e9a4e764aa8999b9ee551b0bc6e/users/afbdf2e06ca249b0b037b28750c444a8/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:28:08.385673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e93fa9-99cd-4df4-81be-8c1c46e28cdf admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:08.412409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e93fa9-99cd-4df4-81be-8c1c46e28cdf admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:08.491814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:08.492790 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:08.493201 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:08.493403 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:08.494336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:08.496166 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:08.517325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1178/2342] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:28:08.520438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063454-ace0-4917-b44f-1c04380efced admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:08.534109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063454-ace0-4917-b44f-1c04380efced admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:08.540979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77503cb2-3a5c-4950-8647-3234c2bbb566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:08.543476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77503cb2-3a5c-4950-8647-3234c2bbb566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:08.543476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77503cb2-3a5c-4950-8647-3234c2bbb566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:08.543476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77503cb2-3a5c-4950-8647-3234c2bbb566 admin admin] PATH_INFO: `/v3/projects/79274e9a4e764aa8999b9ee551b0bc6e/users/afbdf2e06ca249b0b037b28750c444a8/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:08.544700 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77503cb2-3a5c-4950-8647-3234c2bbb566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79274e9a4e764aa8999b9ee551b0bc6e, user_id=afbdf2e06ca249b0b037b28750c444a8, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:08.547548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77503cb2-3a5c-4950-8647-3234c2bbb566 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:08.563390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1165/2343] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:28:08 2022] PUT /identity/v3/projects/79274e9a4e764aa8999b9ee551b0bc6e/users/afbdf2e06ca249b0b037b28750c444a8/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:28:09.074077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:09.077219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:09.092609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfb281-b097-428f-9936-289c3732a504 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:09.207188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:09.208120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:09.208463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:09.208724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] PATH_INFO: `/v3/users/33ac4c1b17484e24a54e95b3fcfc1031` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:09.210847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ac4c1b17484e24a54e95b3fcfc1031)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:09.213159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:09.482141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ac4c1b17484e24a54e95b3fcfc1031'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:09.484260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ac4c1b17484e24a54e95b3fcfc1031'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:09.593083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ac4c1b17484e24a54e95b3fcfc1031'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:09.611498 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ac4c1b17484e24a54e95b3fcfc1031'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:09.612368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ac4c1b17484e24a54e95b3fcfc1031'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:09.651065 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1179/2344] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:28:09 2022] DELETE /identity/v3/users/33ac4c1b17484e24a54e95b3fcfc1031 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:28:09.656044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588d60-257d-4a9b-86f8-1cd13aabd3c2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:09.656562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588d60-257d-4a9b-86f8-1cd13aabd3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:09.656562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588d60-257d-4a9b-86f8-1cd13aabd3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:09.681591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-37588d60-257d-4a9b-86f8-1cd13aabd3c2 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:09.683416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-37588d60-257d-4a9b-86f8-1cd13aabd3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:09.830775 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1166/2345] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:09.842674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:09.846621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:10.048467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df301c1d-ddcf-477e-bad7-d22d80e9e25f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:10.058537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2f644-0e75-4da3-9499-c6bf9e8bbf57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:10.059228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f644-0e75-4da3-9499-c6bf9e8bbf57 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:10.059499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f644-0e75-4da3-9499-c6bf9e8bbf57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:10.059777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2f644-0e75-4da3-9499-c6bf9e8bbf57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:10.082840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2f644-0e75-4da3-9499-c6bf9e8bbf57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:10.084341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2f644-0e75-4da3-9499-c6bf9e8bbf57 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:10.122558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1180/2346] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:28:10.727606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37588d60-257d-4a9b-86f8-1cd13aabd3c2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:10.729999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-37588d60-257d-4a9b-86f8-1cd13aabd3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:10.759952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37588d60-257d-4a9b-86f8-1cd13aabd3c2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:10.780759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:10.781667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:10.781992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:10.782275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] PATH_INFO: `/v3/projects/7063f805431a433bab3249108e0443af` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:10.787542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7063f805431a433bab3249108e0443af)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:10.845283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:11.067697 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7063f805431a433bab3249108e0443af', 'request_id': 'req-b27e3d3e-7672-48c9-9123-723f1cc89f9c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:11.076326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7063f805431a433bab3249108e0443af', 'request_id': 'req-b27e3d3e-7672-48c9-9123-723f1cc89f9c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:11.161961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7063f805431a433bab3249108e0443af', 'request_id': 'req-b27e3d3e-7672-48c9-9123-723f1cc89f9c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:11.180422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] The token cache is being invalidate because project 7063f805431a433bab3249108e0443af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:28:11.180713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:11.188530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b27e3d3e-7672-48c9-9123-723f1cc89f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:11.195606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1167/2347] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:28:10 2022] DELETE /identity/v3/projects/7063f805431a433bab3249108e0443af => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:28:11.202660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3ad0-c710-48b4-ad8a-63f7bc94738f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:11.204138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3ad0-c710-48b4-ad8a-63f7bc94738f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:11.204138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3ad0-c710-48b4-ad8a-63f7bc94738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:11.258813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-c7dc3ad0-c710-48b4-ad8a-63f7bc94738f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:11.260053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c7dc3ad0-c710-48b4-ad8a-63f7bc94738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:11.471073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1181/2348] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:11.487381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc9e3a-3866-428a-a653-4f232bb51231 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:11.487974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc9e3a-3866-428a-a653-4f232bb51231 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:11.488947 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc9e3a-3866-428a-a653-4f232bb51231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:11.527019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6ebc9e3a-3866-428a-a653-4f232bb51231 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:11.528303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6ebc9e3a-3866-428a-a653-4f232bb51231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:11.545813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1168/2349] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:11.550129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc3ad0-c710-48b4-ad8a-63f7bc94738f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:11.568806 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc3ad0-c710-48b4-ad8a-63f7bc94738f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:11.584078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:11.585037 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:11.585435 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:11.585895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:11.586780 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:28:11.587221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:11.589170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:11.613738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1182/2350] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:28:11.615470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc9e3a-3866-428a-a653-4f232bb51231 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:11.629596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc9e3a-3866-428a-a653-4f232bb51231 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:11.635683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:11.636550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:11.636940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:11.637327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:11.638171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:11.640317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:12.076266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1169/2351] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:28:11 2022] POST /identity/v3/projects => generated 373 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:28:12.078753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:12.091755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab57de0-3946-400e-a225-8b1de2f7896f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:12.302431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b3cf1-a2c8-4e80-8b70-77bc7286713f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:12.304055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b3cf1-a2c8-4e80-8b70-77bc7286713f admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:12.304557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b3cf1-a2c8-4e80-8b70-77bc7286713f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:12.304991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b3cf1-a2c8-4e80-8b70-77bc7286713f admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:12.305929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b3cf1-a2c8-4e80-8b70-77bc7286713f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:12.308511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b3cf1-a2c8-4e80-8b70-77bc7286713f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:12.331064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:12.333059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:12.490990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992f162-58ec-4b8f-9f52-5c2122ded42c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:12.496803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '310c5d3ba81044ac8f3f192728cc3e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:12.497471 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:12.497754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:12.498061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:12.502732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:12.519693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:12.521983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:12.537037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1170/2352] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:28:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:28:12.587101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1183/2353] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:28:12 2022] POST /identity/v3/users => generated 287 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:28:12.588113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:12.620600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fe097-bcf1-4ddb-bd13-2750754f2205 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:12.637900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad99c1-b133-4d24-b073-595d59f6e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:12.638624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad99c1-b133-4d24-b073-595d59f6e254 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:12.640126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad99c1-b133-4d24-b073-595d59f6e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:12.640428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad99c1-b133-4d24-b073-595d59f6e254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:12.641372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad99c1-b133-4d24-b073-595d59f6e254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:12.647528 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad99c1-b133-4d24-b073-595d59f6e254 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:12.672444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1171/2354] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:28:12.672633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b3cf1-a2c8-4e80-8b70-77bc7286713f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:12.686658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b3cf1-a2c8-4e80-8b70-77bc7286713f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:12.707850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b1472-427b-49c8-921f-c7b6a12dbece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:12.708552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b1472-427b-49c8-921f-c7b6a12dbece admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:12.708834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b1472-427b-49c8-921f-c7b6a12dbece admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:12.709109 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b1472-427b-49c8-921f-c7b6a12dbece admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:12.709858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b1472-427b-49c8-921f-c7b6a12dbece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:12.711659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b1472-427b-49c8-921f-c7b6a12dbece admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:12.746030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1184/2355] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:28:12.749038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad99c1-b133-4d24-b073-595d59f6e254 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:12.762311 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad99c1-b133-4d24-b073-595d59f6e254 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:12.769994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:12.774349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:12.775153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:12.775986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] PATH_INFO: `/v3/projects/cf41675800ef4cea9d8865f15ea46d2e/users/68ac38391eab4eb99a6db0c2fd0b6b1f/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:12.824064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf41675800ef4cea9d8865f15ea46d2e, user_id=68ac38391eab4eb99a6db0c2fd0b6b1f, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:12.828205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:12.929518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1172/2356] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:28:12 2022] PUT /identity/v3/projects/cf41675800ef4cea9d8865f15ea46d2e/users/68ac38391eab4eb99a6db0c2fd0b6b1f/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:28:12.929713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e9e4c-ffed-482e-9681-ec2fc6baf835 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:12.929713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e9e4c-ffed-482e-9681-ec2fc6baf835 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:12.929713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e9e4c-ffed-482e-9681-ec2fc6baf835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:12.966862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-906e9e4c-ffed-482e-9681-ec2fc6baf835 None None] MFA Rules not processed for user `68ac38391eab4eb99a6db0c2fd0b6b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:12.967903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-906e9e4c-ffed-482e-9681-ec2fc6baf835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:13.056804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1185/2357] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:13.083423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:13.085185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:13.299793 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5373d9-306a-48c8-a16f-09e8817faca3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:13.314759 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90316cc8-0c5d-4bc0-96ae-6cc27456d216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:13.315947 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90316cc8-0c5d-4bc0-96ae-6cc27456d216 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:13.316418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90316cc8-0c5d-4bc0-96ae-6cc27456d216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:13.316862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90316cc8-0c5d-4bc0-96ae-6cc27456d216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:13.335475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90316cc8-0c5d-4bc0-96ae-6cc27456d216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:13.337094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90316cc8-0c5d-4bc0-96ae-6cc27456d216 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:13.360583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1173/2358] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:28:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:28:15.239515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f859a-6460-4165-8c0d-f690c808948a None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:15.240497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f859a-6460-4165-8c0d-f690c808948a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:15.240939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f859a-6460-4165-8c0d-f690c808948a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:15.267269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bb2f859a-6460-4165-8c0d-f690c808948a None None] MFA Rules not processed for user `68ac38391eab4eb99a6db0c2fd0b6b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:15.268769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bb2f859a-6460-4165-8c0d-f690c808948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:15.286361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1186/2359] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:18.078032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4aa3-5bea-4cd9-9991-8bfc0c0e0110 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:18.078548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4aa3-5bea-4cd9-9991-8bfc0c0e0110 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:18.078648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd4aa3-5bea-4cd9-9991-8bfc0c0e0110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:18.125402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bdcd4aa3-5bea-4cd9-9991-8bfc0c0e0110 None None] MFA Rules not processed for user `afbdf2e06ca249b0b037b28750c444a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:18.126819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bdcd4aa3-5bea-4cd9-9991-8bfc0c0e0110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:18.278192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1174/2360] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:18.294040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f859a-6460-4165-8c0d-f690c808948a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:18.295970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bb2f859a-6460-4165-8c0d-f690c808948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:18.387606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f859a-6460-4165-8c0d-f690c808948a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:18.396158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3f786b791b4c8796d251948917ca93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:18.396158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:18.396158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:18.396158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:18.398327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:18.493057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:18.495243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:18.506988 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1187/2361] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:28:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:28:18.909863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284c88f-9e42-40b1-8104-dd0a3ca8aa97 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:18.910470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284c88f-9e42-40b1-8104-dd0a3ca8aa97 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:18.910757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284c88f-9e42-40b1-8104-dd0a3ca8aa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:18.944060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5284c88f-9e42-40b1-8104-dd0a3ca8aa97 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:18.945730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5284c88f-9e42-40b1-8104-dd0a3ca8aa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:19.063000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1175/2362] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:28:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:19.077304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:19.080335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:19.176449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e5ec-5942-44c3-9679-ae7076f425d5 service glance] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:19.182718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072962-1b78-43d9-97bc-c6004410e53a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:19.183706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072962-1b78-43d9-97bc-c6004410e53a service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:19.184218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072962-1b78-43d9-97bc-c6004410e53a service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:19.184663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072962-1b78-43d9-97bc-c6004410e53a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:19.197414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072962-1b78-43d9-97bc-c6004410e53a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:19.199331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072962-1b78-43d9-97bc-c6004410e53a service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:19.221431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1188/2363] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:28:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 13 13:28:28.574058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b352-138b-418d-a617-aca23447664b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:28.574972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b352-138b-418d-a617-aca23447664b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:28.575407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02b352-138b-418d-a617-aca23447664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:28.605601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3c02b352-138b-418d-a617-aca23447664b None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:28.605601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3c02b352-138b-418d-a617-aca23447664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:28.605601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1176/2364] 10.222.0.25 () {58 vars in 967 bytes} [Fri May 13 13:28:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:28.619200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072962-1b78-43d9-97bc-c6004410e53a service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:28.629744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072962-1b78-43d9-97bc-c6004410e53a service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:28.635417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae62f59-e6c7-4279-aaf4-be6a9c797bb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:28.636047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae62f59-e6c7-4279-aaf4-be6a9c797bb7 service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:28.636513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae62f59-e6c7-4279-aaf4-be6a9c797bb7 service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:28.636790 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae62f59-e6c7-4279-aaf4-be6a9c797bb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:28.645530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae62f59-e6c7-4279-aaf4-be6a9c797bb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:28.647174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae62f59-e6c7-4279-aaf4-be6a9c797bb7 service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:28.656173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1189/2365] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:28:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:28:30.746246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3d70-36c8-4b2f-8b9f-9945ab36b105 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:30.746807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3d70-36c8-4b2f-8b9f-9945ab36b105 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:30.746807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3d70-36c8-4b2f-8b9f-9945ab36b105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:30.766322 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-545c3d70-36c8-4b2f-8b9f-9945ab36b105 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:30.767671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-545c3d70-36c8-4b2f-8b9f-9945ab36b105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:30.777949 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1177/2366] 10.222.0.25 () {58 vars in 967 bytes} [Fri May 13 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:30.792465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae62f59-e6c7-4279-aaf4-be6a9c797bb7 service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:30.807053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae62f59-e6c7-4279-aaf4-be6a9c797bb7 service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:30.812421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:30.813180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:30.813674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:30.814082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:30.825243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:30.827842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:30.837324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1190/2367] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:28:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:28:32.850210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96103dd-f45f-488e-9a6c-b9bbd2b09a2d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:32.850745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96103dd-f45f-488e-9a6c-b9bbd2b09a2d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:32.850840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96103dd-f45f-488e-9a6c-b9bbd2b09a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:32.868359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f96103dd-f45f-488e-9a6c-b9bbd2b09a2d None None] MFA Rules not processed for user `015fce3883ab4ee0960565c1a51f520d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:32.869468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f96103dd-f45f-488e-9a6c-b9bbd2b09a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:32.876266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1178/2368] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:28:34.594806 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:34.596818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:34.666998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c22c64-bf2c-499c-a6ca-3316f31b75af service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:34.671928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:34.672906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:34.673260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:34.673718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] PATH_INFO: `/v3/users/1cdee89f8a8040ca944458d3fc6f18eb` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:34.674976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cdee89f8a8040ca944458d3fc6f18eb)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:34.676959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:34.930463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cdee89f8a8040ca944458d3fc6f18eb'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:34.930463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cdee89f8a8040ca944458d3fc6f18eb'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.051850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cdee89f8a8040ca944458d3fc6f18eb'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.060319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cdee89f8a8040ca944458d3fc6f18eb'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.061697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-74a1ad48-a379-48d5-b9e9-fd4f4c628262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cdee89f8a8040ca944458d3fc6f18eb'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.080314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1191/2369] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:28:34 2022] DELETE /identity/v3/users/1cdee89f8a8040ca944458d3fc6f18eb => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:28:35.324468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96103dd-f45f-488e-9a6c-b9bbd2b09a2d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:35.336617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96103dd-f45f-488e-9a6c-b9bbd2b09a2d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:35.397344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:35.398264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:35.398549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:35.398822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] PATH_INFO: `/v3/projects/3ea1460c53954c6a915fc638a2690cbf` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:35.400042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea1460c53954c6a915fc638a2690cbf)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:35.402045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:35.527495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea1460c53954c6a915fc638a2690cbf', 'request_id': 'req-b13bb40c-7e59-48af-b169-32cd39c52f2a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.533254 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea1460c53954c6a915fc638a2690cbf', 'request_id': 'req-b13bb40c-7e59-48af-b169-32cd39c52f2a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.602902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea1460c53954c6a915fc638a2690cbf', 'request_id': 'req-b13bb40c-7e59-48af-b169-32cd39c52f2a'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.615439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] The token cache is being invalidate because project 3ea1460c53954c6a915fc638a2690cbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:28:35.615730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.616375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b13bb40c-7e59-48af-b169-32cd39c52f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:28:35.617952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1179/2370] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:28:35 2022] DELETE /identity/v3/projects/3ea1460c53954c6a915fc638a2690cbf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:28:35.622887 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4083-8fd0-48c7-af7c-0ca193763bbf None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:35.623384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4083-8fd0-48c7-af7c-0ca193763bbf None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:35.623478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f4083-8fd0-48c7-af7c-0ca193763bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:35.640889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f41f4083-8fd0-48c7-af7c-0ca193763bbf None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:35.642199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f41f4083-8fd0-48c7-af7c-0ca193763bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:35.700028 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1192/2371] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:35.708346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d135b-b816-402a-9606-8bfcf1328646 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:35.708635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d135b-b816-402a-9606-8bfcf1328646 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:35.708910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d135b-b816-402a-9606-8bfcf1328646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:35.727728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-c33d135b-b816-402a-9606-8bfcf1328646 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:35.728907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c33d135b-b816-402a-9606-8bfcf1328646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:35.738041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1180/2372] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:35.741719 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f4083-8fd0-48c7-af7c-0ca193763bbf None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:35.751578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f4083-8fd0-48c7-af7c-0ca193763bbf None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:35.756620 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:35.757303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:35.757591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:35.757905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:35.758656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:28:35.758952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:35.760564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:35.774343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1193/2373] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:28:35.779222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d135b-b816-402a-9606-8bfcf1328646 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:35.790047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d135b-b816-402a-9606-8bfcf1328646 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:35.795959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392db7a-1e5a-4c8a-94e0-6f28859081d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:35.796844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392db7a-1e5a-4c8a-94e0-6f28859081d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:35.797313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392db7a-1e5a-4c8a-94e0-6f28859081d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:35.797892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392db7a-1e5a-4c8a-94e0-6f28859081d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:35.798893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392db7a-1e5a-4c8a-94e0-6f28859081d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:35.801182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392db7a-1e5a-4c8a-94e0-6f28859081d9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:35.958402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1181/2374] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:28:35 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:28:35.962695 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:35.973250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826312f8-dfce-4256-8de6-fb70e0ce283f admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:36.026650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45243b-0f7e-4f4a-a663-6234f5c074d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:36.027321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45243b-0f7e-4f4a-a663-6234f5c074d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:36.027610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45243b-0f7e-4f4a-a663-6234f5c074d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:36.027891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45243b-0f7e-4f4a-a663-6234f5c074d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:36.028593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45243b-0f7e-4f4a-a663-6234f5c074d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:36.031121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45243b-0f7e-4f4a-a663-6234f5c074d6 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:36.143017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1194/2375] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:28:35 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:28:36.143533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392db7a-1e5a-4c8a-94e0-6f28859081d9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:36.158381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392db7a-1e5a-4c8a-94e0-6f28859081d9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:36.164079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeaf5e-f3d0-4440-b5ea-0cde46572174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:36.164277 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeaf5e-f3d0-4440-b5ea-0cde46572174 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:36.164549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeaf5e-f3d0-4440-b5ea-0cde46572174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:36.164821 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeaf5e-f3d0-4440-b5ea-0cde46572174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:36.165572 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeeaf5e-f3d0-4440-b5ea-0cde46572174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:36.167221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeeaf5e-f3d0-4440-b5ea-0cde46572174 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:36.181494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1182/2376] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:28:36.182774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45243b-0f7e-4f4a-a663-6234f5c074d6 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:36.192899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45243b-0f7e-4f4a-a663-6234f5c074d6 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:36.202035 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7fb9e-7178-44b7-bc58-3eb568d9527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:36.202738 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7fb9e-7178-44b7-bc58-3eb568d9527e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:36.203244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7fb9e-7178-44b7-bc58-3eb568d9527e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:36.203526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7fb9e-7178-44b7-bc58-3eb568d9527e admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:36.204269 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7fb9e-7178-44b7-bc58-3eb568d9527e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:36.209007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7fb9e-7178-44b7-bc58-3eb568d9527e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:36.224871 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1195/2377] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:28:36.228783 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeaf5e-f3d0-4440-b5ea-0cde46572174 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:36.241596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeaf5e-f3d0-4440-b5ea-0cde46572174 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:36.249084 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:36.250476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:36.251131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:36.251414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] PATH_INFO: `/v3/projects/d3e32e04d1f44e97b903f98c7c2749db/users/8101850dd18f4108ac12ff9487cbbe3e/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:36.273698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e32e04d1f44e97b903f98c7c2749db, user_id=8101850dd18f4108ac12ff9487cbbe3e, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:36.273698 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:36.378473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1183/2378] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:28:36 2022] PUT /identity/v3/projects/d3e32e04d1f44e97b903f98c7c2749db/users/8101850dd18f4108ac12ff9487cbbe3e/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:28:36.383772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84502-4d4f-4223-90e2-9901e6c8bfd0 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:36.384167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84502-4d4f-4223-90e2-9901e6c8bfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:36.384506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f84502-4d4f-4223-90e2-9901e6c8bfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:36.402144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-58f84502-4d4f-4223-90e2-9901e6c8bfd0 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:36.403355 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-58f84502-4d4f-4223-90e2-9901e6c8bfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:36.460443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1196/2379] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:28:36.470851 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:36.472727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:36.530263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7b3b4-b4f2-4afa-902c-1a868f271f38 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:36.537740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:36.537740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:36.537740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:36.537740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:36.548418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:36.550045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:36.571104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1184/2380] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:28:36.707405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5676fd-7c33-412b-8a6c-b83e634b28fd None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:36.707889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5676fd-7c33-412b-8a6c-b83e634b28fd None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:36.707969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5676fd-7c33-412b-8a6c-b83e634b28fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:36.732170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-eb5676fd-7c33-412b-8a6c-b83e634b28fd None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:36.733685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-eb5676fd-7c33-412b-8a6c-b83e634b28fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:36.795469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1197/2381] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:28:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:36.808302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:36.809693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:36.889695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f70cc-0ca3-47a5-b48e-d50c88c6e86f service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:36.895144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3f0a5-54bb-438a-9853-79b87c0f3e5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:36.895796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f0a5-54bb-438a-9853-79b87c0f3e5e service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:36.896079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f0a5-54bb-438a-9853-79b87c0f3e5e service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:36.896517 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f0a5-54bb-438a-9853-79b87c0f3e5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:36.908202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3f0a5-54bb-438a-9853-79b87c0f3e5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:36.913212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3f0a5-54bb-438a-9853-79b87c0f3e5e service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:36.943721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1185/2382] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:28:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:28:43.903548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09bf01-4696-4055-a497-22cbd21d93b0 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:43.904628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09bf01-4696-4055-a497-22cbd21d93b0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:43.905182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09bf01-4696-4055-a497-22cbd21d93b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:43.946864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0b09bf01-4696-4055-a497-22cbd21d93b0 None None] MFA Rules not processed for user `8101850dd18f4108ac12ff9487cbbe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:43.951063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0b09bf01-4696-4055-a497-22cbd21d93b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:44.020313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1198/2383] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:44.024659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3f0a5-54bb-438a-9853-79b87c0f3e5e service swift] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:44.036099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3f0a5-54bb-438a-9853-79b87c0f3e5e service swift] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:44.044834 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a580ef-5ce4-4f90-b4a0-872664a61bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:44.045774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a580ef-5ce4-4f90-b4a0-872664a61bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:44.046088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a580ef-5ce4-4f90-b4a0-872664a61bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:44.046383 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a580ef-5ce4-4f90-b4a0-872664a61bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:44.047066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a580ef-5ce4-4f90-b4a0-872664a61bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:44.048990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a580ef-5ce4-4f90-b4a0-872664a61bf4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:44.156553 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1186/2384] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:28:44 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:28:44.157743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09bf01-4696-4055-a497-22cbd21d93b0 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:44.175817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09bf01-4696-4055-a497-22cbd21d93b0 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:44.239102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b092b-c8e3-4247-b1cb-2fcfb2b12871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:44.239102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b092b-c8e3-4247-b1cb-2fcfb2b12871 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:44.239102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b092b-c8e3-4247-b1cb-2fcfb2b12871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:44.239102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b092b-c8e3-4247-b1cb-2fcfb2b12871 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:44.239825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b092b-c8e3-4247-b1cb-2fcfb2b12871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:44.241583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b092b-c8e3-4247-b1cb-2fcfb2b12871 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:44.344043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1199/2385] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:28:44 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:28:44.351347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a580ef-5ce4-4f90-b4a0-872664a61bf4 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:44.362489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a580ef-5ce4-4f90-b4a0-872664a61bf4 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:44.367475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff78c6d-97c1-45da-b8a7-211b34fd0d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:44.368151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78c6d-97c1-45da-b8a7-211b34fd0d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:44.368449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78c6d-97c1-45da-b8a7-211b34fd0d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:44.368706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78c6d-97c1-45da-b8a7-211b34fd0d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:44.369430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff78c6d-97c1-45da-b8a7-211b34fd0d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:44.371484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff78c6d-97c1-45da-b8a7-211b34fd0d37 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:44.388240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b092b-c8e3-4247-b1cb-2fcfb2b12871 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:44.389582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1187/2386] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:28:44.399723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b092b-c8e3-4247-b1cb-2fcfb2b12871 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:44.405477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ca6fa-877a-40a9-9017-f52da92ba2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:44.406466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ca6fa-877a-40a9-9017-f52da92ba2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:44.406753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ca6fa-877a-40a9-9017-f52da92ba2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:44.407032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ca6fa-877a-40a9-9017-f52da92ba2b3 admin admin] PATH_INFO: `/v3/projects/9cbe2e9981c749b29a3a0d9b2d6d4db3/users/13cac6c04c624feab53852403f56b011/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:44.426214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ca6fa-877a-40a9-9017-f52da92ba2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cbe2e9981c749b29a3a0d9b2d6d4db3, user_id=13cac6c04c624feab53852403f56b011, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:44.429098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ca6fa-877a-40a9-9017-f52da92ba2b3 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:44.563921 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1200/2387] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:28:44 2022] PUT /identity/v3/projects/9cbe2e9981c749b29a3a0d9b2d6d4db3/users/13cac6c04c624feab53852403f56b011/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:28:44.565546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff78c6d-97c1-45da-b8a7-211b34fd0d37 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:44.575722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff78c6d-97c1-45da-b8a7-211b34fd0d37 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:44.647004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d992a-ddd0-49ab-9ea5-34d90bfd2801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:44.647706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d992a-ddd0-49ab-9ea5-34d90bfd2801 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:44.647994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d992a-ddd0-49ab-9ea5-34d90bfd2801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:44.648276 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d992a-ddd0-49ab-9ea5-34d90bfd2801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:44.650147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d992a-ddd0-49ab-9ea5-34d90bfd2801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:44.651832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d992a-ddd0-49ab-9ea5-34d90bfd2801 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:44.676024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1188/2388] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:28:44.676128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ca6fa-877a-40a9-9017-f52da92ba2b3 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:44.692453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ca6fa-877a-40a9-9017-f52da92ba2b3 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:44.697863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:44.699048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:44.699345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:44.699624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] PATH_INFO: `/v3/projects/9cbe2e9981c749b29a3a0d9b2d6d4db3/users/13cac6c04c624feab53852403f56b011/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:44.702251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cbe2e9981c749b29a3a0d9b2d6d4db3, user_id=13cac6c04c624feab53852403f56b011, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:44.706007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:44.724721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1201/2389] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:28:44 2022] PUT /identity/v3/projects/9cbe2e9981c749b29a3a0d9b2d6d4db3/users/13cac6c04c624feab53852403f56b011/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:28:52.739151 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7769-ad83-4f2c-8556-c1c414ae6e61 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:52.739659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7769-ad83-4f2c-8556-c1c414ae6e61 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:52.739771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7769-ad83-4f2c-8556-c1c414ae6e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:52.783774 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-107f7769-ad83-4f2c-8556-c1c414ae6e61 None None] MFA Rules not processed for user `13cac6c04c624feab53852403f56b011`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:52.785940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-107f7769-ad83-4f2c-8556-c1c414ae6e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:52.872628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1189/2390] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:28:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:52.969689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:52.972828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:53.053353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beacf7-80eb-4f30-8013-3cf5d2287b1d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:53.058742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca497c-151e-4ff3-a9a2-745d7bf2a528 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:53.059383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca497c-151e-4ff3-a9a2-745d7bf2a528 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:53.059664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca497c-151e-4ff3-a9a2-745d7bf2a528 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:53.059938 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca497c-151e-4ff3-a9a2-745d7bf2a528 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:53.113207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca497c-151e-4ff3-a9a2-745d7bf2a528 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:53.114758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca497c-151e-4ff3-a9a2-745d7bf2a528 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:53.123608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1202/2391] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:28:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:28:59.342481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e978237-32a3-4dcc-bcea-2b06b8ca436d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:59.342985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e978237-32a3-4dcc-bcea-2b06b8ca436d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:59.343080 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e978237-32a3-4dcc-bcea-2b06b8ca436d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:59.363261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1e978237-32a3-4dcc-bcea-2b06b8ca436d None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:28:59.364693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1e978237-32a3-4dcc-bcea-2b06b8ca436d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:28:59.422929 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1190/2392] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:28:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:28:59.433688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca497c-151e-4ff3-a9a2-745d7bf2a528 service nova] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:28:59.528745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca497c-151e-4ff3-a9a2-745d7bf2a528 service nova] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:28:59.536772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90df90-421c-44c2-8d7b-4330aac2bcf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:28:59.537573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90df90-421c-44c2-8d7b-4330aac2bcf2 service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:28:59.537979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90df90-421c-44c2-8d7b-4330aac2bcf2 service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:28:59.538383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90df90-421c-44c2-8d7b-4330aac2bcf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:28:59.548393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90df90-421c-44c2-8d7b-4330aac2bcf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:28:59.550276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90df90-421c-44c2-8d7b-4330aac2bcf2 service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:28:59.558742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1203/2393] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:28:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:29:00.680301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2c29a-d609-4b9e-91d1-3f721bb1fea7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:00.680301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2c29a-d609-4b9e-91d1-3f721bb1fea7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:00.680301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2c29a-d609-4b9e-91d1-3f721bb1fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:00.694309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-50d2c29a-d609-4b9e-91d1-3f721bb1fea7 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:29:00.695717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-50d2c29a-d609-4b9e-91d1-3f721bb1fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:00.704008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1191/2394] 10.222.0.25 () {58 vars in 967 bytes} [Fri May 13 13:29:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:29:00.716588 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90df90-421c-44c2-8d7b-4330aac2bcf2 service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:29:00.728478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90df90-421c-44c2-8d7b-4330aac2bcf2 service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:29:00.735039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53beb4-bda6-482e-9e19-0a04605a2bfd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:29:00.735716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53beb4-bda6-482e-9e19-0a04605a2bfd service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:00.736052 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53beb4-bda6-482e-9e19-0a04605a2bfd service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:00.736328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53beb4-bda6-482e-9e19-0a04605a2bfd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:00.745744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53beb4-bda6-482e-9e19-0a04605a2bfd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:29:00.747490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53beb4-bda6-482e-9e19-0a04605a2bfd service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:29:00.756948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1204/2395] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:29:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:29:01.610828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb577c52-0b76-40f0-b793-2a10cc6e2ef0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:01.611114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb577c52-0b76-40f0-b793-2a10cc6e2ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:01.611392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb577c52-0b76-40f0-b793-2a10cc6e2ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:01.628476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-eb577c52-0b76-40f0-b793-2a10cc6e2ef0 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:29:01.629648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-eb577c52-0b76-40f0-b793-2a10cc6e2ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:01.637050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1192/2396] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:29:01.649576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53beb4-bda6-482e-9e19-0a04605a2bfd service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:29:01.660653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53beb4-bda6-482e-9e19-0a04605a2bfd service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:29:01.665762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb64513-432d-47ab-9910-c7c2b52460f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:29:01.666450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb64513-432d-47ab-9910-c7c2b52460f0 service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:01.666730 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb64513-432d-47ab-9910-c7c2b52460f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:01.667116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb64513-432d-47ab-9910-c7c2b52460f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:01.675305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb64513-432d-47ab-9910-c7c2b52460f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:29:01.676927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb64513-432d-47ab-9910-c7c2b52460f0 service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:29:01.685904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1205/2397] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:29:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:29:02.331841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df05a-e8bb-4b44-b1e5-210cd76ff709 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:02.332299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df05a-e8bb-4b44-b1e5-210cd76ff709 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:02.332402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df05a-e8bb-4b44-b1e5-210cd76ff709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:02.349727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-db4df05a-e8bb-4b44-b1e5-210cd76ff709 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:29:02.350956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-db4df05a-e8bb-4b44-b1e5-210cd76ff709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:02.359657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1193/2398] 10.222.0.25 () {58 vars in 967 bytes} [Fri May 13 13:29:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:29:02.372064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb64513-432d-47ab-9910-c7c2b52460f0 service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:29:02.383023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb64513-432d-47ab-9910-c7c2b52460f0 service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:29:02.388038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e9b7d-228b-4540-b048-1899c2f6d0f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:29:02.388700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e9b7d-228b-4540-b048-1899c2f6d0f0 service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:02.388985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e9b7d-228b-4540-b048-1899c2f6d0f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:02.389267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e9b7d-228b-4540-b048-1899c2f6d0f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:02.397710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e9b7d-228b-4540-b048-1899c2f6d0f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:29:02.399507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e9b7d-228b-4540-b048-1899c2f6d0f0 service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:29:02.408118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1206/2399] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:29:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:29:05.314285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4df05a-e8bb-4b44-b1e5-210cd76ff709 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:29:05.315848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-db4df05a-e8bb-4b44-b1e5-210cd76ff709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:05.386539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4df05a-e8bb-4b44-b1e5-210cd76ff709 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:29:05.391297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:29:05.392178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:05.392458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:05.392733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] PATH_INFO: `/v3/users/68ac38391eab4eb99a6db0c2fd0b6b1f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:05.393987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ac38391eab4eb99a6db0c2fd0b6b1f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:29:05.395933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:29:05.699716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ac38391eab4eb99a6db0c2fd0b6b1f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:05.700395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ac38391eab4eb99a6db0c2fd0b6b1f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:05.700998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ac38391eab4eb99a6db0c2fd0b6b1f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:05.708478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ac38391eab4eb99a6db0c2fd0b6b1f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:05.863557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ac38391eab4eb99a6db0c2fd0b6b1f'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:05.888358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1194/2400] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:29:05 2022] DELETE /identity/v3/users/68ac38391eab4eb99a6db0c2fd0b6b1f => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:29:05.893061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689537-ce3e-4cbd-8aca-f0da2f065c1c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:05.893549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689537-ce3e-4cbd-8aca-f0da2f065c1c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:05.893646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689537-ce3e-4cbd-8aca-f0da2f065c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:05.911826 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-4a689537-ce3e-4cbd-8aca-f0da2f065c1c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:29:05.913050 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4a689537-ce3e-4cbd-8aca-f0da2f065c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:05.982753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1207/2401] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:29:05.992233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:29:05.997687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:06.069180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8562fe1-56cd-4702-a91c-6a13ac103f37 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:29:06.079115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af70ab1-5776-4549-a0eb-b12c5af41c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:29:06.079115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af70ab1-5776-4549-a0eb-b12c5af41c77 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:06.079115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af70ab1-5776-4549-a0eb-b12c5af41c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:06.079115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af70ab1-5776-4549-a0eb-b12c5af41c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:06.102510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af70ab1-5776-4549-a0eb-b12c5af41c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:29:06.104139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af70ab1-5776-4549-a0eb-b12c5af41c77 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:29:06.115302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1195/2402] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:29:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:29:06.398980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689537-ce3e-4cbd-8aca-f0da2f065c1c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:29:06.400610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4a689537-ce3e-4cbd-8aca-f0da2f065c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:06.414182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689537-ce3e-4cbd-8aca-f0da2f065c1c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:29:06.420825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:29:06.422147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:06.422720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:06.423153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] PATH_INFO: `/v3/projects/cf41675800ef4cea9d8865f15ea46d2e` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:06.424868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf41675800ef4cea9d8865f15ea46d2e)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:29:06.428068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:29:06.594084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf41675800ef4cea9d8865f15ea46d2e', 'request_id': 'req-d5dfc7c3-4557-4500-87cf-0602923b3cb3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:06.600624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf41675800ef4cea9d8865f15ea46d2e', 'request_id': 'req-d5dfc7c3-4557-4500-87cf-0602923b3cb3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:06.698020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf41675800ef4cea9d8865f15ea46d2e', 'request_id': 'req-d5dfc7c3-4557-4500-87cf-0602923b3cb3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:06.709969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] The token cache is being invalidate because project cf41675800ef4cea9d8865f15ea46d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:29:06.710259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:06.710983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:29:06.712524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1208/2403] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:29:06 2022] DELETE /identity/v3/projects/cf41675800ef4cea9d8865f15ea46d2e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:29:10.601042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c46ff-7af2-4301-b043-60dfe3166000 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:10.601595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c46ff-7af2-4301-b043-60dfe3166000 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:10.601595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c46ff-7af2-4301-b043-60dfe3166000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:10.618983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-917c46ff-7af2-4301-b043-60dfe3166000 None None] MFA Rules not processed for user `15932bb7b3044a41942b4e0edecebf10`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:29:10.620206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-917c46ff-7af2-4301-b043-60dfe3166000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:10.677192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1196/2404] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:29:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:29:16.054511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:29:16.056129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:16.152202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfc7c3-4557-4500-87cf-0602923b3cb3 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:29:16.156810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f15f98-3abb-485a-9d2f-1bba9b4a5b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:29:16.164616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15f98-3abb-485a-9d2f-1bba9b4a5b6b service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:16.164616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15f98-3abb-485a-9d2f-1bba9b4a5b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:16.164616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15f98-3abb-485a-9d2f-1bba9b4a5b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:16.164616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-71f15f98-3abb-485a-9d2f-1bba9b4a5b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:16.229657 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f15f98-3abb-485a-9d2f-1bba9b4a5b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:29:16.231183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f15f98-3abb-485a-9d2f-1bba9b4a5b6b service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:29:16.241529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1209/2405] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:29:16.560086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65656c0a-907e-4e0b-b529-2e98a128da7d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:16.560687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65656c0a-907e-4e0b-b529-2e98a128da7d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:16.560794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65656c0a-907e-4e0b-b529-2e98a128da7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:16.595286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-65656c0a-907e-4e0b-b529-2e98a128da7d None None] MFA Rules not processed for user `8101850dd18f4108ac12ff9487cbbe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:29:16.597132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-65656c0a-907e-4e0b-b529-2e98a128da7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:16.668282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1197/2406] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:29:32.155260 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba67d2-3da8-4980-ad5a-5b2ef4e4ee9f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:32.155727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba67d2-3da8-4980-ad5a-5b2ef4e4ee9f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:32.155845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba67d2-3da8-4980-ad5a-5b2ef4e4ee9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:32.173313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1fba67d2-3da8-4980-ad5a-5b2ef4e4ee9f None None] MFA Rules not processed for user `f56c908ada31445e85ec1dfee8e2ebdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:29:32.174874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1fba67d2-3da8-4980-ad5a-5b2ef4e4ee9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:32.227961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1210/2407] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:29:41.153847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65656c0a-907e-4e0b-b529-2e98a128da7d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:29:41.155352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-65656c0a-907e-4e0b-b529-2e98a128da7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:41.167798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65656c0a-907e-4e0b-b529-2e98a128da7d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:29:41.172061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:29:41.173004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:41.173452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:41.173944 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:41.176409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:41.246765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:29:41.248264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:29:41.258687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1198/2408] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:29:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:29:42.998289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600878-d222-4d29-899d-73ebc0413fca None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:29:42.999318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600878-d222-4d29-899d-73ebc0413fca None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:29:42.999420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600878-d222-4d29-899d-73ebc0413fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:29:43.016965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-46600878-d222-4d29-899d-73ebc0413fca None None] MFA Rules not processed for user `08b9fbf87cfa417fbea857ae602a140e`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:29:43.018323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-46600878-d222-4d29-899d-73ebc0413fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:29:43.070246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1211/2409] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:29:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:30:12.410689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:12.419255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fad80d-20b4-4261-9cab-031eff3e2501 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:12.423453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:12.424068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:12.434965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:12.434965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:12.434965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:12.498843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:12.500878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:12.512183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1199/2410] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:30:13.705919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46600878-d222-4d29-899d-73ebc0413fca None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:13.715386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46600878-d222-4d29-899d-73ebc0413fca None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:13.718978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:13.720170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:13.720667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:13.721094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:13.723656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:13.738929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:13.740643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:13.751992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1212/2411] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:25.182480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:25.183915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:25.197453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8634588-69e5-4de4-8563-ff36eb585678 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:25.202907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:25.203540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:25.203822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:25.204619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:25.206340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:25.261100 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:25.262782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:25.272072 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1200/2412] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:28.845862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:28.847897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:28.924103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceb7ce-ac0b-40e0-8b89-a742ae375821 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:28.929343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:28.930282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:28.930565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:28.930892 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] PATH_INFO: `/v3/users/15932bb7b3044a41942b4e0edecebf10` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:28.932098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] RBAC: Authorizing `identity:delete_user(user_id=15932bb7b3044a41942b4e0edecebf10)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:28.934098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:29.173161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15932bb7b3044a41942b4e0edecebf10'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.174009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15932bb7b3044a41942b4e0edecebf10'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.243902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15932bb7b3044a41942b4e0edecebf10'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.249232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15932bb7b3044a41942b4e0edecebf10'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.249954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15932bb7b3044a41942b4e0edecebf10'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.267997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1213/2413] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:30:28 2022] DELETE /identity/v3/users/15932bb7b3044a41942b4e0edecebf10 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:30:29.643414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:29.658190 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd269c-3dbf-4125-b7e3-709321fda84a service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:29.712033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:29.712936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:29.713218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:29.713495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] PATH_INFO: `/v3/projects/b361133457c04e2da4c4158a45bbcf19` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:29.714751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b361133457c04e2da4c4158a45bbcf19)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:29.716737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:29.859688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b361133457c04e2da4c4158a45bbcf19', 'request_id': 'req-000a5769-11c9-4190-b55b-a566ded2df38'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.865709 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b361133457c04e2da4c4158a45bbcf19', 'request_id': 'req-000a5769-11c9-4190-b55b-a566ded2df38'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.927720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b361133457c04e2da4c4158a45bbcf19', 'request_id': 'req-000a5769-11c9-4190-b55b-a566ded2df38'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.950376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] The token cache is being invalidate because project b361133457c04e2da4c4158a45bbcf19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:30:29.950376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.950376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-000a5769-11c9-4190-b55b-a566ded2df38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:30:29.950376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1201/2414] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:30:29 2022] DELETE /identity/v3/projects/b361133457c04e2da4c4158a45bbcf19 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:30:30.517390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:30.518873 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:30.574239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001147e2-1142-499a-8966-99d8c471724e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:30.623130 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:30.626122 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:30.626122 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:30.626122 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:30.627188 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:30.683413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:30.684900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:30.694229 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1214/2415] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:33.763780 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d673cf-5960-4364-8268-46a99dc44003 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:33.764264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d673cf-5960-4364-8268-46a99dc44003 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:33.764472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d673cf-5960-4364-8268-46a99dc44003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:33.782535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d3d673cf-5960-4364-8268-46a99dc44003 None None] MFA Rules not processed for user `233f5342cf334c7689cb9e71f4fbaab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:30:33.783738 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d3d673cf-5960-4364-8268-46a99dc44003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:33.837875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1202/2416] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:30:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:30:34.133472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:34.135021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:34.190753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55b03-7c17-4ec4-9a61-12241fdcab95 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:34.194451 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:34.195111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:34.195424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:34.195700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:34.197529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:34.207258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:34.219123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:34.219123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1215/2417] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:34.671207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d673cf-5960-4364-8268-46a99dc44003 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:34.673308 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d3d673cf-5960-4364-8268-46a99dc44003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:34.745426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d673cf-5960-4364-8268-46a99dc44003 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:34.750458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '310c5d3ba81044ac8f3f192728cc3e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:34.751106 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:34.751390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:34.751668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:34.753413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:34.764425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:34.766038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:34.776115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1203/2418] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:41.092533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:41.096017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:41.108254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5848-af63-4b68-ac83-926dff7fad2b service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:41.113212 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:41.114000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:41.114352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:41.114847 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:41.116894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:41.128625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:41.130540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:41.141407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1216/2419] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:42.031500 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:42.033992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:42.119545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0cdac-1054-471e-8c84-198f6b240ba9 service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:42.129456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:42.130245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:42.130537 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:42.130845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:42.132988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:42.148162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:42.150211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:42.171057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1204/2420] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:48.602195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:48.605064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:48.620236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56193d-3e3e-4708-8e1a-65ede6d43541 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:48.626004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:48.626732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:48.627265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:48.627558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:48.629375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:48.641545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:48.643208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:48.654882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1217/2421] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:49.364817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:49.376877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e28c12-fd00-4ffd-9219-035104419e0a service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:49.382633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:49.383273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:49.383560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:49.383836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:49.385583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:49.443905 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:49.445513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:49.455467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1205/2422] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:30:57.048318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:30:57.057756 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf61e6c-f5ce-4a67-bfe8-e79550208032 service nova] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:30:57.061713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:30:57.062468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:30:57.062879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:30:57.063170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:30:57.064958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:30:57.129352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:30:57.130855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:30:57.141948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1218/2423] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:31:01.699624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:01.701654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:01.717267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcaf3e-86dd-40f0-824f-10130ae3e043 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:01.725267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:01.727889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:01.727889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:01.727889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] PATH_INFO: `/v3/users/f56c908ada31445e85ec1dfee8e2ebdb` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:01.730345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56c908ada31445e85ec1dfee8e2ebdb)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:01.730345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:01.935189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56c908ada31445e85ec1dfee8e2ebdb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:01.935781 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56c908ada31445e85ec1dfee8e2ebdb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:01.936390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56c908ada31445e85ec1dfee8e2ebdb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:01.943878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56c908ada31445e85ec1dfee8e2ebdb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:02.035813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-ad7e2088-aadd-4cef-960b-3ebe1a263aa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56c908ada31445e85ec1dfee8e2ebdb'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:02.053733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1206/2424] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:31:01 2022] DELETE /identity/v3/users/f56c908ada31445e85ec1dfee8e2ebdb => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:31:02.295556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:02.305416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece8d1e-3b24-4678-b76f-bdfeb268071d service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:02.387753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:02.390622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:02.391451 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:02.392067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] PATH_INFO: `/v3/projects/e25902914d244bfeaa038f24838c28ca` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:02.393307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e25902914d244bfeaa038f24838c28ca)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:02.395256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:02.539964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e25902914d244bfeaa038f24838c28ca', 'request_id': 'req-267adbb5-22e4-445a-99c9-e572a43e040f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:02.549580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e25902914d244bfeaa038f24838c28ca', 'request_id': 'req-267adbb5-22e4-445a-99c9-e572a43e040f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:02.647113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e25902914d244bfeaa038f24838c28ca', 'request_id': 'req-267adbb5-22e4-445a-99c9-e572a43e040f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:02.658493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] The token cache is being invalidate because project e25902914d244bfeaa038f24838c28ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:31:02.658804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:02.659426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-267adbb5-22e4-445a-99c9-e572a43e040f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:02.660950 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1219/2425] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:31:02 2022] DELETE /identity/v3/projects/e25902914d244bfeaa038f24838c28ca => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:31:02.666083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabf08c-a413-4853-9b3c-5316f6f754cf None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:02.666380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabf08c-a413-4853-9b3c-5316f6f754cf None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:02.666666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabf08c-a413-4853-9b3c-5316f6f754cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:02.684463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ddabf08c-a413-4853-9b3c-5316f6f754cf None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:02.685664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ddabf08c-a413-4853-9b3c-5316f6f754cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:02.743382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1207/2426] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:02.751140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196bfd6-cadd-4247-9816-f0e713c6ee0b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:02.751402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196bfd6-cadd-4247-9816-f0e713c6ee0b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:02.751684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196bfd6-cadd-4247-9816-f0e713c6ee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:02.769622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6196bfd6-cadd-4247-9816-f0e713c6ee0b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:02.770899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6196bfd6-cadd-4247-9816-f0e713c6ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:02.778804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1220/2427] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:02.783469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabf08c-a413-4853-9b3c-5316f6f754cf None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:02.792987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabf08c-a413-4853-9b3c-5316f6f754cf None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:02.798271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:02.798918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:02.799202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:02.799505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:02.800230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:31:02.800540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:02.802183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:02.814412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1208/2428] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:31:02.818614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196bfd6-cadd-4247-9816-f0e713c6ee0b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:02.827702 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196bfd6-cadd-4247-9816-f0e713c6ee0b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:02.833021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beff83a-9c59-4db7-877d-4f71440881cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:02.833681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff83a-9c59-4db7-877d-4f71440881cf admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:02.834009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff83a-9c59-4db7-877d-4f71440881cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:02.834307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff83a-9c59-4db7-877d-4f71440881cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:02.835002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beff83a-9c59-4db7-877d-4f71440881cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:02.836955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beff83a-9c59-4db7-877d-4f71440881cf admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:02.969118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1221/2429] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:31:02 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:31:02.973542 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:02.983365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bbe9b-8807-4757-8813-7c93e891b946 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:03.038141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d2276-053f-4646-a321-da77c4f9d60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:03.038809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d2276-053f-4646-a321-da77c4f9d60b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:03.039090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d2276-053f-4646-a321-da77c4f9d60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:03.039365 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d2276-053f-4646-a321-da77c4f9d60b admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:03.040045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d2276-053f-4646-a321-da77c4f9d60b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:03.042005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d2276-053f-4646-a321-da77c4f9d60b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:03.132837 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1209/2430] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:31:02 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:31:03.137195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beff83a-9c59-4db7-877d-4f71440881cf admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:03.148063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beff83a-9c59-4db7-877d-4f71440881cf admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:03.152720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe7337-020a-446c-8373-dbdaf0adc0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:03.153377 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe7337-020a-446c-8373-dbdaf0adc0ac admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:03.153663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe7337-020a-446c-8373-dbdaf0adc0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:03.154029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe7337-020a-446c-8373-dbdaf0adc0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:03.154788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe7337-020a-446c-8373-dbdaf0adc0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:03.156395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe7337-020a-446c-8373-dbdaf0adc0ac admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:03.167641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1222/2431] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:31:03.171673 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d2276-053f-4646-a321-da77c4f9d60b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:03.182508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d2276-053f-4646-a321-da77c4f9d60b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:03.187370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43d5a1-fd96-452a-9d1f-22c574926d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:03.188665 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43d5a1-fd96-452a-9d1f-22c574926d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:03.188999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43d5a1-fd96-452a-9d1f-22c574926d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:03.189432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43d5a1-fd96-452a-9d1f-22c574926d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:03.190810 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43d5a1-fd96-452a-9d1f-22c574926d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:03.193836 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43d5a1-fd96-452a-9d1f-22c574926d93 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:03.204355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1210/2432] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:31:03.209015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe7337-020a-446c-8373-dbdaf0adc0ac admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:03.218884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe7337-020a-446c-8373-dbdaf0adc0ac admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:03.223499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:03.224371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:03.224655 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:03.224935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] PATH_INFO: `/v3/projects/1e72f39546704606927bcefdc0f13235/users/79dbde2a65224927af4bb5e77090496c/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:03.239346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e72f39546704606927bcefdc0f13235, user_id=79dbde2a65224927af4bb5e77090496c, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:03.242062 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:03.321185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1223/2433] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:31:03 2022] PUT /identity/v3/projects/1e72f39546704606927bcefdc0f13235/users/79dbde2a65224927af4bb5e77090496c/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:31:03.326756 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd4b30-6350-4031-9804-0edb778e3a3f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:03.327048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd4b30-6350-4031-9804-0edb778e3a3f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:03.327328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd4b30-6350-4031-9804-0edb778e3a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:03.345252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-31cd4b30-6350-4031-9804-0edb778e3a3f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:03.346461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-31cd4b30-6350-4031-9804-0edb778e3a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:03.403079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1211/2434] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:03.413232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:03.414951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:03.469596 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d1c92-a73e-4efa-b221-99acda16ec9d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:03.473345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:03.474025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:03.474309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:03.474609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:03.484242 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:03.485749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:03.495377 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1224/2435] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:31:09.554398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2638b-ec46-4d6f-a928-25b286cc39d1 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:09.554930 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2638b-ec46-4d6f-a928-25b286cc39d1 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:09.555053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2638b-ec46-4d6f-a928-25b286cc39d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:09.582021 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6df2638b-ec46-4d6f-a928-25b286cc39d1 None None] MFA Rules not processed for user `79dbde2a65224927af4bb5e77090496c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:09.583232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6df2638b-ec46-4d6f-a928-25b286cc39d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:09.645055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1212/2436] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:09.659683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:09.661091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:09.732713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d51c6-2f17-4751-8442-599295be7728 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:09.738778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db24d2e-9cae-4c65-954a-7a914367430f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:09.739430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db24d2e-9cae-4c65-954a-7a914367430f service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:09.739708 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db24d2e-9cae-4c65-954a-7a914367430f service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:09.739983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db24d2e-9cae-4c65-954a-7a914367430f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:09.748163 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db24d2e-9cae-4c65-954a-7a914367430f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:09.749811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db24d2e-9cae-4c65-954a-7a914367430f service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:09.758949 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1225/2437] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:31:12.612298 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda27a11-88a6-4b7e-a6e8-7d7e9797d9f8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:12.612943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda27a11-88a6-4b7e-a6e8-7d7e9797d9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:12.612943 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda27a11-88a6-4b7e-a6e8-7d7e9797d9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:12.632863 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-fda27a11-88a6-4b7e-a6e8-7d7e9797d9f8 None None] MFA Rules not processed for user `79dbde2a65224927af4bb5e77090496c`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:12.634519 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fda27a11-88a6-4b7e-a6e8-7d7e9797d9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:12.643474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1213/2438] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:17.950584 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b223-1a38-4295-bfe2-32b1ae716510 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:17.951114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b223-1a38-4295-bfe2-32b1ae716510 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:17.951114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b223-1a38-4295-bfe2-32b1ae716510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:17.970431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1825b223-1a38-4295-bfe2-32b1ae716510 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:17.972326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1825b223-1a38-4295-bfe2-32b1ae716510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:18.038320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1226/2439] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:18.228480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda27a11-88a6-4b7e-a6e8-7d7e9797d9f8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:18.229988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fda27a11-88a6-4b7e-a6e8-7d7e9797d9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:18.301289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda27a11-88a6-4b7e-a6e8-7d7e9797d9f8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:18.306452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2094627-97c4-49f2-93a6-ab158c790bd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:18.307102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094627-97c4-49f2-93a6-ab158c790bd4 service swift] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:18.307390 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094627-97c4-49f2-93a6-ab158c790bd4 service swift] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:18.307669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094627-97c4-49f2-93a6-ab158c790bd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:18.316611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094627-97c4-49f2-93a6-ab158c790bd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:18.318249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094627-97c4-49f2-93a6-ab158c790bd4 service swift] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:18.326621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1214/2440] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:31:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:31:19.391700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b223-1a38-4295-bfe2-32b1ae716510 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:19.393330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1825b223-1a38-4295-bfe2-32b1ae716510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:19.434089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b223-1a38-4295-bfe2-32b1ae716510 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:19.440957 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:19.442200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:19.442784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:19.443384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:19.446610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:19.502992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:19.508073 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:19.545457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1227/2441] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:31:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:31:24.461720 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9d8f4-524f-44fb-9f15-ce81b8712c84 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:24.462392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9d8f4-524f-44fb-9f15-ce81b8712c84 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:24.462392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9d8f4-524f-44fb-9f15-ce81b8712c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:24.484222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-86b9d8f4-524f-44fb-9f15-ce81b8712c84 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:24.485755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-86b9d8f4-524f-44fb-9f15-ce81b8712c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:24.503167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1215/2442] 10.222.0.25 () {58 vars in 967 bytes} [Fri May 13 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:24.562361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:24.569636 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ee732-da90-43d8-819b-928b57ff6fb2 service nova] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:24.579227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7939fac-5691-46fb-8ad4-b88744c17023 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:24.579903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939fac-5691-46fb-8ad4-b88744c17023 service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:24.580473 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939fac-5691-46fb-8ad4-b88744c17023 service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:24.581250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939fac-5691-46fb-8ad4-b88744c17023 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:24.590635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7939fac-5691-46fb-8ad4-b88744c17023 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:24.592252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7939fac-5691-46fb-8ad4-b88744c17023 service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:24.606821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1228/2443] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:31:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:31:36.603172 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9d8f4-524f-44fb-9f15-ce81b8712c84 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:36.604614 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-86b9d8f4-524f-44fb-9f15-ce81b8712c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:36.619578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9d8f4-524f-44fb-9f15-ce81b8712c84 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:36.624223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:36.625108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:36.625396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:36.625678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] PATH_INFO: `/v3/users/015fce3883ab4ee0960565c1a51f520d` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:36.627022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] RBAC: Authorizing `identity:delete_user(user_id=015fce3883ab4ee0960565c1a51f520d)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:36.628962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:36.855246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '015fce3883ab4ee0960565c1a51f520d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:36.855906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '015fce3883ab4ee0960565c1a51f520d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:36.856565 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '015fce3883ab4ee0960565c1a51f520d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:36.863409 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '015fce3883ab4ee0960565c1a51f520d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:36.966850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '015fce3883ab4ee0960565c1a51f520d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:36.984843 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1216/2444] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:31:36 2022] DELETE /identity/v3/users/015fce3883ab4ee0960565c1a51f520d => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:31:37.231527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7939fac-5691-46fb-8ad4-b88744c17023 service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:37.243396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7939fac-5691-46fb-8ad4-b88744c17023 service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:37.303427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:37.304305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:37.304593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:37.304873 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] PATH_INFO: `/v3/projects/1503b2654ff7475b98032519919ae481` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:37.306154 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1503b2654ff7475b98032519919ae481)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:37.308140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:37.493091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1503b2654ff7475b98032519919ae481', 'request_id': 'req-033b35c5-1b19-4bfb-bbea-1000bda1218e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.499098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1503b2654ff7475b98032519919ae481', 'request_id': 'req-033b35c5-1b19-4bfb-bbea-1000bda1218e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.561222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1503b2654ff7475b98032519919ae481', 'request_id': 'req-033b35c5-1b19-4bfb-bbea-1000bda1218e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.572971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] The token cache is being invalidate because project 1503b2654ff7475b98032519919ae481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:31:37.573259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.573964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.575622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1229/2445] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:31:37 2022] DELETE /identity/v3/projects/1503b2654ff7475b98032519919ae481 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:31:37.579610 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:37.581405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:37.594549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a267c19-8693-416f-9cf3-054d97bc594c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:37.652879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:37.653764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:37.654085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:37.654377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] PATH_INFO: `/v3/users/afbdf2e06ca249b0b037b28750c444a8` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:37.655560 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=afbdf2e06ca249b0b037b28750c444a8)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:37.657531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:37.856085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afbdf2e06ca249b0b037b28750c444a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.856785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afbdf2e06ca249b0b037b28750c444a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.857393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afbdf2e06ca249b0b037b28750c444a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.864772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afbdf2e06ca249b0b037b28750c444a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.915478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c5209874-6b39-4d4e-8ec5-7688213f0b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afbdf2e06ca249b0b037b28750c444a8'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:37.932957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1217/2446] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:31:37 2022] DELETE /identity/v3/users/afbdf2e06ca249b0b037b28750c444a8 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:31:38.271344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:38.272801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:38.290865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b35c5-1b19-4bfb-bbea-1000bda1218e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:38.350913 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:38.351786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:38.352101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:38.352376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] PATH_INFO: `/v3/projects/79274e9a4e764aa8999b9ee551b0bc6e` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:38.355971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79274e9a4e764aa8999b9ee551b0bc6e)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:38.355971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:38.578690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79274e9a4e764aa8999b9ee551b0bc6e', 'request_id': 'req-f55ad24c-f113-498a-b503-42d8e43fa880'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:38.584459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79274e9a4e764aa8999b9ee551b0bc6e', 'request_id': 'req-f55ad24c-f113-498a-b503-42d8e43fa880'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:38.692811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79274e9a4e764aa8999b9ee551b0bc6e', 'request_id': 'req-f55ad24c-f113-498a-b503-42d8e43fa880'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:38.704231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] The token cache is being invalidate because project 79274e9a4e764aa8999b9ee551b0bc6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:31:38.704516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:38.705148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f55ad24c-f113-498a-b503-42d8e43fa880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:31:38.706973 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1230/2447] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:31:38 2022] DELETE /identity/v3/projects/79274e9a4e764aa8999b9ee551b0bc6e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:31:38.714576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49927be7-1f31-41d7-b559-3eefe5866504 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:38.715041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49927be7-1f31-41d7-b559-3eefe5866504 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:38.715162 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49927be7-1f31-41d7-b559-3eefe5866504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:38.733637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-49927be7-1f31-41d7-b559-3eefe5866504 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:38.735343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-49927be7-1f31-41d7-b559-3eefe5866504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:38.795377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1218/2448] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:38.857938 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabadd1d-ead1-4c8b-a72a-0c0ffe4feeb6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:38.858226 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabadd1d-ead1-4c8b-a72a-0c0ffe4feeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:38.858505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabadd1d-ead1-4c8b-a72a-0c0ffe4feeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:38.876608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-fabadd1d-ead1-4c8b-a72a-0c0ffe4feeb6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:38.877815 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-fabadd1d-ead1-4c8b-a72a-0c0ffe4feeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:38.885718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1231/2449] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:38.889749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49927be7-1f31-41d7-b559-3eefe5866504 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:38.899393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49927be7-1f31-41d7-b559-3eefe5866504 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:38.904018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:38.904667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:38.904955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:38.905238 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:38.905997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:31:38.906803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:38.908438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:38.919640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1219/2450] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:31:38.924178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabadd1d-ead1-4c8b-a72a-0c0ffe4feeb6 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:38.933050 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabadd1d-ead1-4c8b-a72a-0c0ffe4feeb6 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:38.937604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341875d-6662-4c88-8ccc-d1aa7fc9bcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:38.938276 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341875d-6662-4c88-8ccc-d1aa7fc9bcfa admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:38.938556 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341875d-6662-4c88-8ccc-d1aa7fc9bcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:38.938831 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341875d-6662-4c88-8ccc-d1aa7fc9bcfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:38.939505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341875d-6662-4c88-8ccc-d1aa7fc9bcfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:38.941398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341875d-6662-4c88-8ccc-d1aa7fc9bcfa admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:39.077091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1232/2451] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:31:38 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:31:39.082109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:39.092418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b337ae-ab8d-437f-9a8e-cd9b4618d846 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:39.163557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d7d5e-adb1-4071-af2d-f8bfc7d29155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:39.163557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7d5e-adb1-4071-af2d-f8bfc7d29155 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:39.163557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7d5e-adb1-4071-af2d-f8bfc7d29155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:39.163917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d7d5e-adb1-4071-af2d-f8bfc7d29155 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:39.164389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d7d5e-adb1-4071-af2d-f8bfc7d29155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:39.166369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d7d5e-adb1-4071-af2d-f8bfc7d29155 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:39.330187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1220/2452] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:31:39 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:31:39.331696 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341875d-6662-4c88-8ccc-d1aa7fc9bcfa admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:39.342457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341875d-6662-4c88-8ccc-d1aa7fc9bcfa admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:39.348025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798b986-7729-45fe-99f5-dd653c7a86ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:39.348739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798b986-7729-45fe-99f5-dd653c7a86ec admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:39.349018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798b986-7729-45fe-99f5-dd653c7a86ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:39.349287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798b986-7729-45fe-99f5-dd653c7a86ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:39.350036 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798b986-7729-45fe-99f5-dd653c7a86ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:39.351641 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798b986-7729-45fe-99f5-dd653c7a86ec admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:39.363391 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1233/2453] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:31:39.367396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d7d5e-adb1-4071-af2d-f8bfc7d29155 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:39.377631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d7d5e-adb1-4071-af2d-f8bfc7d29155 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:39.382525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b954af5-1100-4fce-be69-62cbafa816f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:39.383207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b954af5-1100-4fce-be69-62cbafa816f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:39.383489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b954af5-1100-4fce-be69-62cbafa816f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:39.383762 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b954af5-1100-4fce-be69-62cbafa816f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:39.384483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b954af5-1100-4fce-be69-62cbafa816f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:39.386143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b954af5-1100-4fce-be69-62cbafa816f5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:39.397204 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1221/2454] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:31:39.401327 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798b986-7729-45fe-99f5-dd653c7a86ec admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:39.411893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798b986-7729-45fe-99f5-dd653c7a86ec admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:39.416767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:39.417693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:39.418038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:39.418320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] PATH_INFO: `/v3/projects/0b1cc4c699be4edfa243bc7dde91cd51/users/9c6a8bd3014349a3aa60c9471f3c8234/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:39.434010 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1cc4c699be4edfa243bc7dde91cd51, user_id=9c6a8bd3014349a3aa60c9471f3c8234, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:39.437106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:39.584347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1234/2455] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:31:39 2022] PUT /identity/v3/projects/0b1cc4c699be4edfa243bc7dde91cd51/users/9c6a8bd3014349a3aa60c9471f3c8234/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:31:39.589480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc3d38-0eb1-4b45-a52d-0cf59232e4c8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:39.593465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc3d38-0eb1-4b45-a52d-0cf59232e4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:39.593465 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc3d38-0eb1-4b45-a52d-0cf59232e4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:39.608060 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-13fc3d38-0eb1-4b45-a52d-0cf59232e4c8 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:39.609953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-13fc3d38-0eb1-4b45-a52d-0cf59232e4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:39.669378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1222/2456] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:39.679695 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:39.681406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:39.739555 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d175e8b-6b64-48a8-a7e5-373f5814e3b8 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:39.743824 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:39.744471 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:39.744765 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:39.745027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:39.755152 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:39.756582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:39.766048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1235/2457] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:31:39.982741 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284b1c3-8fd4-4467-9b80-16b93a6b0012 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:39.983243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284b1c3-8fd4-4467-9b80-16b93a6b0012 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:39.983338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284b1c3-8fd4-4467-9b80-16b93a6b0012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:40.002207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a284b1c3-8fd4-4467-9b80-16b93a6b0012 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:40.003400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a284b1c3-8fd4-4467-9b80-16b93a6b0012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:40.061878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1223/2458] 10.222.0.25 () {58 vars in 1009 bytes} [Fri May 13 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:40.074778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:40.076304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:40.152119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167bb86-482d-4f3b-9849-8b6b2bb13b4a service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:40.164040 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5fd7-979d-43b4-867a-dc96525222c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:40.164723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5fd7-979d-43b4-867a-dc96525222c5 service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:40.165015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5fd7-979d-43b4-867a-dc96525222c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:40.165353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5fd7-979d-43b4-867a-dc96525222c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:40.176573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5fd7-979d-43b4-867a-dc96525222c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:40.178579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5fd7-979d-43b4-867a-dc96525222c5 service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:40.188094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1236/2459] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:31:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:31:47.396689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797088-6509-4140-b118-3516d5a370bd None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:47.397847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797088-6509-4140-b118-3516d5a370bd None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:47.398671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c797088-6509-4140-b118-3516d5a370bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:47.440406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-3c797088-6509-4140-b118-3516d5a370bd None None] MFA Rules not processed for user `9c6a8bd3014349a3aa60c9471f3c8234`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:47.443232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3c797088-6509-4140-b118-3516d5a370bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:47.525105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1224/2460] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:47.529888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5fd7-979d-43b4-867a-dc96525222c5 service swift] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:47.541493 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5fd7-979d-43b4-867a-dc96525222c5 service swift] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:47.546223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b262f-9d67-4f53-b9ec-3e9f5e663642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:47.546923 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b262f-9d67-4f53-b9ec-3e9f5e663642 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:47.547209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b262f-9d67-4f53-b9ec-3e9f5e663642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:47.547482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b262f-9d67-4f53-b9ec-3e9f5e663642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:47.548194 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b262f-9d67-4f53-b9ec-3e9f5e663642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:47.550302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b262f-9d67-4f53-b9ec-3e9f5e663642 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:48.578700 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1237/2461] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:31:47 2022] POST /identity/v3/projects => generated 373 bytes in 1046 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:31:48.580575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c797088-6509-4140-b118-3516d5a370bd None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:48.594764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c797088-6509-4140-b118-3516d5a370bd None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:48.693567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aaa06-37b9-47eb-a381-0876624fd194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:48.694895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aaa06-37b9-47eb-a381-0876624fd194 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:48.695732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aaa06-37b9-47eb-a381-0876624fd194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:48.696518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aaa06-37b9-47eb-a381-0876624fd194 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:48.698179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119aaa06-37b9-47eb-a381-0876624fd194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:48.703394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119aaa06-37b9-47eb-a381-0876624fd194 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:48.876955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1225/2462] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:31:48 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:31:48.885019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b262f-9d67-4f53-b9ec-3e9f5e663642 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:48.904190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b262f-9d67-4f53-b9ec-3e9f5e663642 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:48.909612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f085db2-7b76-4b51-b3fb-a284de750e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:48.910407 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f085db2-7b76-4b51-b3fb-a284de750e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:48.910696 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f085db2-7b76-4b51-b3fb-a284de750e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:48.910980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f085db2-7b76-4b51-b3fb-a284de750e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:48.911725 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f085db2-7b76-4b51-b3fb-a284de750e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:48.913371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f085db2-7b76-4b51-b3fb-a284de750e02 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:48.925305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1238/2463] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:31:48.934736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aaa06-37b9-47eb-a381-0876624fd194 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:48.946076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aaa06-37b9-47eb-a381-0876624fd194 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:48.951076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f2c92-2b72-43af-ba32-26916091a253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:48.951987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2c92-2b72-43af-ba32-26916091a253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:48.952301 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2c92-2b72-43af-ba32-26916091a253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:48.952581 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2c92-2b72-43af-ba32-26916091a253 admin admin] PATH_INFO: `/v3/projects/81a2c144876d4dfc8b4d4ed76426fdbf/users/f72e83f97c884e4b93df99535e66a3cc/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:48.970335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f2c92-2b72-43af-ba32-26916091a253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a2c144876d4dfc8b4d4ed76426fdbf, user_id=f72e83f97c884e4b93df99535e66a3cc, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:48.980423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f2c92-2b72-43af-ba32-26916091a253 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:49.099849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1226/2464] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:31:48 2022] PUT /identity/v3/projects/81a2c144876d4dfc8b4d4ed76426fdbf/users/f72e83f97c884e4b93df99535e66a3cc/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:31:49.108340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f085db2-7b76-4b51-b3fb-a284de750e02 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:49.115677 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f085db2-7b76-4b51-b3fb-a284de750e02 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:49.193924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db7eec-1634-46aa-8bfd-d547c4c08ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:49.194606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db7eec-1634-46aa-8bfd-d547c4c08ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:49.194899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db7eec-1634-46aa-8bfd-d547c4c08ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:49.195179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db7eec-1634-46aa-8bfd-d547c4c08ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:49.195904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db7eec-1634-46aa-8bfd-d547c4c08ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:49.201557 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db7eec-1634-46aa-8bfd-d547c4c08ebf admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:49.214941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1239/2465] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:31:49.219314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f2c92-2b72-43af-ba32-26916091a253 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:49.234399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f2c92-2b72-43af-ba32-26916091a253 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:49.242029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8553871-b133-4217-94e5-924f3b4580e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:49.243077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553871-b133-4217-94e5-924f3b4580e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:49.243432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553871-b133-4217-94e5-924f3b4580e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:49.243792 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8553871-b133-4217-94e5-924f3b4580e7 admin admin] PATH_INFO: `/v3/projects/81a2c144876d4dfc8b4d4ed76426fdbf/users/f72e83f97c884e4b93df99535e66a3cc/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:49.246027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8553871-b133-4217-94e5-924f3b4580e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a2c144876d4dfc8b4d4ed76426fdbf, user_id=f72e83f97c884e4b93df99535e66a3cc, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:49.251294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8553871-b133-4217-94e5-924f3b4580e7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:49.263076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1227/2466] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:31:49 2022] PUT /identity/v3/projects/81a2c144876d4dfc8b4d4ed76426fdbf/users/f72e83f97c884e4b93df99535e66a3cc/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:31:55.112403 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd86a-e1c2-4c50-9ce5-ce22d7579394 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:55.113313 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd86a-e1c2-4c50-9ce5-ce22d7579394 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:55.113664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd86a-e1c2-4c50-9ce5-ce22d7579394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:55.133165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-a57cd86a-e1c2-4c50-9ce5-ce22d7579394 None None] MFA Rules not processed for user `63d11bc6b12949f28957146b7d66a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:55.134400 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a57cd86a-e1c2-4c50-9ce5-ce22d7579394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:55.156494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314bde2-8e61-4bd6-a992-543e66d6b43e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:55.157029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314bde2-8e61-4bd6-a992-543e66d6b43e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:55.157521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314bde2-8e61-4bd6-a992-543e66d6b43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:55.189420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-6314bde2-8e61-4bd6-a992-543e66d6b43e None None] MFA Rules not processed for user `f72e83f97c884e4b93df99535e66a3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:55.190869 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6314bde2-8e61-4bd6-a992-543e66d6b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:55.213236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1240/2467] 10.222.0.25 () {58 vars in 967 bytes} [Fri May 13 13:31:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:31:55.238882 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cd86a-e1c2-4c50-9ce5-ce22d7579394 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:55.286371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1228/2468] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:31:55.300817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314bde2-8e61-4bd6-a992-543e66d6b43e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:31:55.302204 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6314bde2-8e61-4bd6-a992-543e66d6b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:55.335303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cd86a-e1c2-4c50-9ce5-ce22d7579394 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:55.340947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631cb1d-d69a-4219-8e46-631c38ce882f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96f6d41c529444fba00bb09ac4224eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:55.341595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631cb1d-d69a-4219-8e46-631c38ce882f service swift] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:55.342019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631cb1d-d69a-4219-8e46-631c38ce882f service swift] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:55.342187 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631cb1d-d69a-4219-8e46-631c38ce882f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:55.350929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631cb1d-d69a-4219-8e46-631c38ce882f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:55.352534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631cb1d-d69a-4219-8e46-631c38ce882f service swift] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:55.361745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1241/2469] 10.222.0.25 () {60 vars in 1384 bytes} [Fri May 13 13:31:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 13 13:31:55.432521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314bde2-8e61-4bd6-a992-543e66d6b43e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:31:55.439051 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bde165-e110-4921-bd1a-ef32ebae2dbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:31:55.439907 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bde165-e110-4921-bd1a-ef32ebae2dbf service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:55.440371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bde165-e110-4921-bd1a-ef32ebae2dbf service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:55.441455 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bde165-e110-4921-bd1a-ef32ebae2dbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:55.501583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bde165-e110-4921-bd1a-ef32ebae2dbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:31:55.503306 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bde165-e110-4921-bd1a-ef32ebae2dbf service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:31:55.513001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1229/2470] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:31:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:31:58.300002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64908d46-b1d7-4426-acf9-ce856c4e9bb1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:31:58.300464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64908d46-b1d7-4426-acf9-ce856c4e9bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:31:58.300559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64908d46-b1d7-4426-acf9-ce856c4e9bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:31:58.319184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-64908d46-b1d7-4426-acf9-ce856c4e9bb1 None None] MFA Rules not processed for user `9c6a8bd3014349a3aa60c9471f3c8234`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:31:58.320375 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-64908d46-b1d7-4426-acf9-ce856c4e9bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:31:58.327710 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1242/2471] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:08.283116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e283-fa7c-497e-a7f2-a329d0b1352f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:08.283658 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e283-fa7c-497e-a7f2-a329d0b1352f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:08.284038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e283-fa7c-497e-a7f2-a329d0b1352f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:08.314685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5c10e283-fa7c-497e-a7f2-a329d0b1352f None None] MFA Rules not processed for user `8101850dd18f4108ac12ff9487cbbe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:08.316062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5c10e283-fa7c-497e-a7f2-a329d0b1352f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:08.388376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1230/2472] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:14.841385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64908d46-b1d7-4426-acf9-ce856c4e9bb1 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:14.898935 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64908d46-b1d7-4426-acf9-ce856c4e9bb1 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:14.902601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:14.903251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:14.903539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:14.909576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:14.909576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:14.974499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:14.976275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:14.990055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1243/2473] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:15.579336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10e283-fa7c-497e-a7f2-a329d0b1352f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:15.590359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10e283-fa7c-497e-a7f2-a329d0b1352f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:15.594237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:15.595057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:15.595345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:15.595623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:15.598855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:15.608899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:15.610888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:15.618702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1231/2474] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:20.020027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:20.022307 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:20.040547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85c4fe-4d08-4fa0-91d9-be13bb9f5850 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:20.045453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:20.046476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:20.047544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:20.047890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] PATH_INFO: `/v3/users/233f5342cf334c7689cb9e71f4fbaab3` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:20.049320 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] RBAC: Authorizing `identity:delete_user(user_id=233f5342cf334c7689cb9e71f4fbaab3)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:20.051430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:20.320148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '233f5342cf334c7689cb9e71f4fbaab3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:20.320807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '233f5342cf334c7689cb9e71f4fbaab3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:20.383905 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '233f5342cf334c7689cb9e71f4fbaab3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:20.393110 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '233f5342cf334c7689cb9e71f4fbaab3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:20.393681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-19e083b4-6a1e-4cf1-8bba-023b7c0affda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '233f5342cf334c7689cb9e71f4fbaab3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:20.420416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1244/2475] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:32:20 2022] DELETE /identity/v3/users/233f5342cf334c7689cb9e71f4fbaab3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:20.699394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:20.713031 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750b2b7-4c77-4d6a-8778-51359d07068a service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:20.787416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:20.788292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:20.788575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:20.788852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] PATH_INFO: `/v3/projects/915a7ff1f6d64480a08c38e762da4134` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:20.790328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=915a7ff1f6d64480a08c38e762da4134)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:20.792278 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:21.012434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '915a7ff1f6d64480a08c38e762da4134', 'request_id': 'req-8d739eab-7626-4d45-9ce0-46382ee124ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:21.018426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '915a7ff1f6d64480a08c38e762da4134', 'request_id': 'req-8d739eab-7626-4d45-9ce0-46382ee124ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:21.099702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '915a7ff1f6d64480a08c38e762da4134', 'request_id': 'req-8d739eab-7626-4d45-9ce0-46382ee124ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:21.113638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] The token cache is being invalidate because project 915a7ff1f6d64480a08c38e762da4134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:32:21.113968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:21.115157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8d739eab-7626-4d45-9ce0-46382ee124ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:21.116715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1232/2476] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:32:20 2022] DELETE /identity/v3/projects/915a7ff1f6d64480a08c38e762da4134 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:21.121804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b8aaa-fca0-4849-97cf-9d93fccd00b3 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.122343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b8aaa-fca0-4849-97cf-9d93fccd00b3 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.123123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b8aaa-fca0-4849-97cf-9d93fccd00b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.142138 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b48b8aaa-fca0-4849-97cf-9d93fccd00b3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:21.149811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b48b8aaa-fca0-4849-97cf-9d93fccd00b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:21.217497 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1245/2477] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:21.225879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525b1db-3e33-4bd5-aeb8-0c0aa11101f3 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.226173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525b1db-3e33-4bd5-aeb8-0c0aa11101f3 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.226453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525b1db-3e33-4bd5-aeb8-0c0aa11101f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.247732 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5525b1db-3e33-4bd5-aeb8-0c0aa11101f3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:21.249428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5525b1db-3e33-4bd5-aeb8-0c0aa11101f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:21.258363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1233/2478] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:21.262692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b8aaa-fca0-4849-97cf-9d93fccd00b3 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:21.273316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b8aaa-fca0-4849-97cf-9d93fccd00b3 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:21.280523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:21.281413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.281701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.282016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.282750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:32:21.283056 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:21.285412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:21.298096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1246/2479] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:32:21.303785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525b1db-3e33-4bd5-aeb8-0c0aa11101f3 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:21.324782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525b1db-3e33-4bd5-aeb8-0c0aa11101f3 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:21.330419 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606e8bc-3a99-4018-ab6e-333a79cde563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:21.331651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606e8bc-3a99-4018-ab6e-333a79cde563 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.332347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606e8bc-3a99-4018-ab6e-333a79cde563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.333052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606e8bc-3a99-4018-ab6e-333a79cde563 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.338411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606e8bc-3a99-4018-ab6e-333a79cde563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:21.340514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606e8bc-3a99-4018-ab6e-333a79cde563 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:21.483762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:21.484750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1234/2480] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:32:21 2022] POST /identity/v3/projects => generated 393 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:21.496200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393f1e1-da5f-4831-961c-14e0a71eaed3 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:21.563940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519261e0-2879-4c47-9744-bcbaa1fef23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:21.564610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519261e0-2879-4c47-9744-bcbaa1fef23a admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.564897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519261e0-2879-4c47-9744-bcbaa1fef23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.565173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519261e0-2879-4c47-9744-bcbaa1fef23a admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.565896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519261e0-2879-4c47-9744-bcbaa1fef23a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:21.567825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519261e0-2879-4c47-9744-bcbaa1fef23a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:21.681420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1247/2481] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:32:21 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:32:21.685599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606e8bc-3a99-4018-ab6e-333a79cde563 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:21.696450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606e8bc-3a99-4018-ab6e-333a79cde563 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:21.701015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d26d3-a959-4178-9ac3-3b41b999155e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:21.701682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d26d3-a959-4178-9ac3-3b41b999155e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.701996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d26d3-a959-4178-9ac3-3b41b999155e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.702281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d26d3-a959-4178-9ac3-3b41b999155e admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.703043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d26d3-a959-4178-9ac3-3b41b999155e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:21.704650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d26d3-a959-4178-9ac3-3b41b999155e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:21.715420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1235/2482] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:21.719915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519261e0-2879-4c47-9744-bcbaa1fef23a admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:21.730528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519261e0-2879-4c47-9744-bcbaa1fef23a admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:21.735908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05cf8-1329-4092-9e91-02054a9aef93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:21.736582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05cf8-1329-4092-9e91-02054a9aef93 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.736869 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05cf8-1329-4092-9e91-02054a9aef93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.737148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05cf8-1329-4092-9e91-02054a9aef93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.737913 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b05cf8-1329-4092-9e91-02054a9aef93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:21.739529 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b05cf8-1329-4092-9e91-02054a9aef93 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:21.760563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1248/2483] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:21.765052 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d26d3-a959-4178-9ac3-3b41b999155e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:21.783045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d26d3-a959-4178-9ac3-3b41b999155e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:21.797310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4a273-827e-4afb-8f73-f313ebf84ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:21.797310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a273-827e-4afb-8f73-f313ebf84ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.797310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a273-827e-4afb-8f73-f313ebf84ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.797310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a273-827e-4afb-8f73-f313ebf84ecb admin admin] PATH_INFO: `/v3/projects/c06c49fcd6c2491aa675e9c50ed8c2e3/users/cfe4ee2523ff4c448b14f31cc714c050/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.817568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4a273-827e-4afb-8f73-f313ebf84ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c06c49fcd6c2491aa675e9c50ed8c2e3, user_id=cfe4ee2523ff4c448b14f31cc714c050, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:21.822951 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4a273-827e-4afb-8f73-f313ebf84ecb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:21.909913 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1236/2484] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:21 2022] PUT /identity/v3/projects/c06c49fcd6c2491aa675e9c50ed8c2e3/users/cfe4ee2523ff4c448b14f31cc714c050/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:21.923448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59646ba-c83c-4ecc-87f9-84512bfb25cd None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:21.923448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59646ba-c83c-4ecc-87f9-84512bfb25cd None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:21.923448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59646ba-c83c-4ecc-87f9-84512bfb25cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:21.955902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b59646ba-c83c-4ecc-87f9-84512bfb25cd None None] MFA Rules not processed for user `cfe4ee2523ff4c448b14f31cc714c050`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:21.955902 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b59646ba-c83c-4ecc-87f9-84512bfb25cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:22.030402 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1249/2485] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:22.032231 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4a273-827e-4afb-8f73-f313ebf84ecb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:22.100629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4a273-827e-4afb-8f73-f313ebf84ecb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:22.105822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c003141-e76d-4b63-b52f-fab07a0afe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:22.106063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c003141-e76d-4b63-b52f-fab07a0afe37 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.106347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c003141-e76d-4b63-b52f-fab07a0afe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.106622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c003141-e76d-4b63-b52f-fab07a0afe37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.107405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c003141-e76d-4b63-b52f-fab07a0afe37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:22.109901 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c003141-e76d-4b63-b52f-fab07a0afe37 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:22.166537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f52be4-2c8a-4c7c-bb54-29429e2d71c3 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.167140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f52be4-2c8a-4c7c-bb54-29429e2d71c3 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.167140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f52be4-2c8a-4c7c-bb54-29429e2d71c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.185308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-30f52be4-2c8a-4c7c-bb54-29429e2d71c3 None None] MFA Rules not processed for user `79dbde2a65224927af4bb5e77090496c`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:22.186536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-30f52be4-2c8a-4c7c-bb54-29429e2d71c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:22.191789 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1237/2486] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:32:22 2022] POST /identity/v3/projects => generated 393 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:32:22.195957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c003141-e76d-4b63-b52f-fab07a0afe37 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:22.253308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1250/2487] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:22.268299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c003141-e76d-4b63-b52f-fab07a0afe37 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:22.273412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84380ed-6ccc-44ca-96f7-9a971af88c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:22.274195 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84380ed-6ccc-44ca-96f7-9a971af88c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.274550 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84380ed-6ccc-44ca-96f7-9a971af88c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.275012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84380ed-6ccc-44ca-96f7-9a971af88c9d admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.275889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84380ed-6ccc-44ca-96f7-9a971af88c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:22.278630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84380ed-6ccc-44ca-96f7-9a971af88c9d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:22.397171 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1238/2488] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:32:22 2022] POST /identity/v3/users => generated 303 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:32:22.401279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f52be4-2c8a-4c7c-bb54-29429e2d71c3 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:22.413131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f52be4-2c8a-4c7c-bb54-29429e2d71c3 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:22.418153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735bbdb-5d6f-4fb6-8920-195a08aceea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:22.419103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735bbdb-5d6f-4fb6-8920-195a08aceea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.419577 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735bbdb-5d6f-4fb6-8920-195a08aceea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.420022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735bbdb-5d6f-4fb6-8920-195a08aceea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.421000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735bbdb-5d6f-4fb6-8920-195a08aceea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:22.422741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735bbdb-5d6f-4fb6-8920-195a08aceea4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:22.442842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1251/2489] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:22.447277 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84380ed-6ccc-44ca-96f7-9a971af88c9d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:22.459683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84380ed-6ccc-44ca-96f7-9a971af88c9d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:22.464090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7d70c-e641-4060-b013-d2e14b1b8360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:22.464865 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7d70c-e641-4060-b013-d2e14b1b8360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.465156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7d70c-e641-4060-b013-d2e14b1b8360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.465440 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7d70c-e641-4060-b013-d2e14b1b8360 admin admin] PATH_INFO: `/v3/projects/1316e71c7d10477c826edd1c757f0a9b/users/33421d33d46c4fc2be664e0cb4a5f0e4/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.489994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7d70c-e641-4060-b013-d2e14b1b8360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1316e71c7d10477c826edd1c757f0a9b, user_id=33421d33d46c4fc2be664e0cb4a5f0e4, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:22.492661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7d70c-e641-4060-b013-d2e14b1b8360 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:22.631510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1239/2490] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:22 2022] PUT /identity/v3/projects/1316e71c7d10477c826edd1c757f0a9b/users/33421d33d46c4fc2be664e0cb4a5f0e4/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:22.635617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735bbdb-5d6f-4fb6-8920-195a08aceea4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:22.646644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735bbdb-5d6f-4fb6-8920-195a08aceea4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:22.709430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50c0f2-0205-4b54-a0ba-2a55395f28aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:22.710392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c0f2-0205-4b54-a0ba-2a55395f28aa admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.710833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c0f2-0205-4b54-a0ba-2a55395f28aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.711217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c0f2-0205-4b54-a0ba-2a55395f28aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.712972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50c0f2-0205-4b54-a0ba-2a55395f28aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:22.714839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50c0f2-0205-4b54-a0ba-2a55395f28aa admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:22.727360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1252/2491] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:22.731568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7d70c-e641-4060-b013-d2e14b1b8360 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:22.743454 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7d70c-e641-4060-b013-d2e14b1b8360 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:22.748625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:22.749526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.750452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.750764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] PATH_INFO: `/v3/projects/1316e71c7d10477c826edd1c757f0a9b/users/33421d33d46c4fc2be664e0cb4a5f0e4/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.752811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1316e71c7d10477c826edd1c757f0a9b, user_id=33421d33d46c4fc2be664e0cb4a5f0e4, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:22.756023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:22.773088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1240/2492] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:22 2022] PUT /identity/v3/projects/1316e71c7d10477c826edd1c757f0a9b/users/33421d33d46c4fc2be664e0cb4a5f0e4/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:22.783582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c534f2-0918-4362-a5d7-196a70b835ba None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.783889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c534f2-0918-4362-a5d7-196a70b835ba None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.784175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c534f2-0918-4362-a5d7-196a70b835ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.812087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e0c534f2-0918-4362-a5d7-196a70b835ba None None] MFA Rules not processed for user `33421d33d46c4fc2be664e0cb4a5f0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:22.813893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e0c534f2-0918-4362-a5d7-196a70b835ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:22.889218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1253/2493] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:32:22.903024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:22.904717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:22.973199 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b338-5b20-4109-8c35-92bd623c7e40 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:22.976987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7113d4-5abe-4f73-9622-9409470927ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:22.977772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7113d4-5abe-4f73-9622-9409470927ef service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:22.978107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7113d4-5abe-4f73-9622-9409470927ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:22.978388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7113d4-5abe-4f73-9622-9409470927ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:22.989654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7113d4-5abe-4f73-9622-9409470927ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:22.991318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7113d4-5abe-4f73-9622-9409470927ef service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:23.004602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1241/2494] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:23.064044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c534f2-0918-4362-a5d7-196a70b835ba None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:23.065866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e0c534f2-0918-4362-a5d7-196a70b835ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:23.079105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c534f2-0918-4362-a5d7-196a70b835ba None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:23.088060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7169b10-aef7-464b-affe-4054eb073592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:23.088060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169b10-aef7-464b-affe-4054eb073592 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:23.088060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169b10-aef7-464b-affe-4054eb073592 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:23.088060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169b10-aef7-464b-affe-4054eb073592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:23.143429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7169b10-aef7-464b-affe-4054eb073592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:23.144999 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7169b10-aef7-464b-affe-4054eb073592 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:23.154189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1254/2495] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:27.835501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7113d4-5abe-4f73-9622-9409470927ef service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:27.897662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7113d4-5abe-4f73-9622-9409470927ef service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:27.902267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:27.903141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:27.903425 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:27.903710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] PATH_INFO: `/v3/users/cfe4ee2523ff4c448b14f31cc714c050` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:27.904912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe4ee2523ff4c448b14f31cc714c050)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:27.907467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:28.076813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe4ee2523ff4c448b14f31cc714c050'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:28.077476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe4ee2523ff4c448b14f31cc714c050'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:28.078110 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe4ee2523ff4c448b14f31cc714c050'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:28.085486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe4ee2523ff4c448b14f31cc714c050'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:28.138029 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe4ee2523ff4c448b14f31cc714c050'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:28.157241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1242/2496] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:32:27 2022] DELETE /identity/v3/users/cfe4ee2523ff4c448b14f31cc714c050 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:28.162112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb050-0646-4dc5-a9c8-b83de4884435 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:28.162605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb050-0646-4dc5-a9c8-b83de4884435 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:28.162701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb050-0646-4dc5-a9c8-b83de4884435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:28.179285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-e33bb050-0646-4dc5-a9c8-b83de4884435 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:28.180528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e33bb050-0646-4dc5-a9c8-b83de4884435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:28.237987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1255/2497] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:28.247621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:28.249008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:28.303323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a51351-98a8-4d83-a506-f8909a7109d7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:28.306952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b871-76bc-44d3-871c-63893ae17dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:28.307590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b871-76bc-44d3-871c-63893ae17dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:28.307866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b871-76bc-44d3-871c-63893ae17dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:28.308166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b871-76bc-44d3-871c-63893ae17dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:28.317147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2b871-76bc-44d3-871c-63893ae17dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:28.318625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2b871-76bc-44d3-871c-63893ae17dc4 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:28.328666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1243/2498] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:28.716304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb050-0646-4dc5-a9c8-b83de4884435 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:28.717872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-e33bb050-0646-4dc5-a9c8-b83de4884435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:28.729907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb050-0646-4dc5-a9c8-b83de4884435 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:28.735185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:28.736057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:28.736337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:28.736612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] PATH_INFO: `/v3/projects/c06c49fcd6c2491aa675e9c50ed8c2e3` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:28.737880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c06c49fcd6c2491aa675e9c50ed8c2e3)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:28.739800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:28.958757 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c06c49fcd6c2491aa675e9c50ed8c2e3', 'request_id': 'req-65e35e7d-7d31-4790-a372-2f5d6f6fae91'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:28.964540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c06c49fcd6c2491aa675e9c50ed8c2e3', 'request_id': 'req-65e35e7d-7d31-4790-a372-2f5d6f6fae91'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.046304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c06c49fcd6c2491aa675e9c50ed8c2e3', 'request_id': 'req-65e35e7d-7d31-4790-a372-2f5d6f6fae91'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.058162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] The token cache is being invalidate because project c06c49fcd6c2491aa675e9c50ed8c2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:32:29.058450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.059890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.060891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1256/2499] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:32:28 2022] DELETE /identity/v3/projects/c06c49fcd6c2491aa675e9c50ed8c2e3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:29.064766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b871-76bc-44d3-871c-63893ae17dc4 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:29.078295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b871-76bc-44d3-871c-63893ae17dc4 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:29.142459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:29.143361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:29.143650 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:29.143928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] PATH_INFO: `/v3/users/33421d33d46c4fc2be664e0cb4a5f0e4` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:29.145140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33421d33d46c4fc2be664e0cb4a5f0e4)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:29.147114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:29.373065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33421d33d46c4fc2be664e0cb4a5f0e4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.374070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33421d33d46c4fc2be664e0cb4a5f0e4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.375223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33421d33d46c4fc2be664e0cb4a5f0e4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.382119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33421d33d46c4fc2be664e0cb4a5f0e4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.458449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-c8b85d6d-8dbe-4440-bba3-3173994d25f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33421d33d46c4fc2be664e0cb4a5f0e4'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:29.478804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1244/2500] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:32:29 2022] DELETE /identity/v3/users/33421d33d46c4fc2be664e0cb4a5f0e4 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:32:29.946874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:29.948319 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:29.962167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e35e7d-7d31-4790-a372-2f5d6f6fae91 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:30.066779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:30.067970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.068417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.068881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] PATH_INFO: `/v3/projects/1316e71c7d10477c826edd1c757f0a9b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.070334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1316e71c7d10477c826edd1c757f0a9b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:30.072542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:30.182612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1316e71c7d10477c826edd1c757f0a9b', 'request_id': 'req-1d3858dc-9f36-4dfa-9881-410826a00981'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:30.189227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1316e71c7d10477c826edd1c757f0a9b', 'request_id': 'req-1d3858dc-9f36-4dfa-9881-410826a00981'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:30.297840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1316e71c7d10477c826edd1c757f0a9b', 'request_id': 'req-1d3858dc-9f36-4dfa-9881-410826a00981'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:30.309838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] The token cache is being invalidate because project 1316e71c7d10477c826edd1c757f0a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:32:30.310207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:30.310919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1d3858dc-9f36-4dfa-9881-410826a00981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:30.313094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1257/2501] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:32:29 2022] DELETE /identity/v3/projects/1316e71c7d10477c826edd1c757f0a9b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:30.317966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c486-f81b-4dbe-84cd-3a520c0ad538 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.318443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c486-f81b-4dbe-84cd-3a520c0ad538 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.318559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c486-f81b-4dbe-84cd-3a520c0ad538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.338461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-be62c486-f81b-4dbe-84cd-3a520c0ad538 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:30.339671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-be62c486-f81b-4dbe-84cd-3a520c0ad538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:30.399061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1245/2502] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:30.407507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1000459-4983-4139-9b23-6af73c817e44 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.407794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1000459-4983-4139-9b23-6af73c817e44 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.408071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1000459-4983-4139-9b23-6af73c817e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.425959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d1000459-4983-4139-9b23-6af73c817e44 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:30.427186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d1000459-4983-4139-9b23-6af73c817e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:30.435623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1258/2503] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:30.440131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62c486-f81b-4dbe-84cd-3a520c0ad538 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:30.449589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62c486-f81b-4dbe-84cd-3a520c0ad538 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:30.454713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:30.455497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.458590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.458590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.458590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:32:30.458590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:30.459038 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:30.470755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1246/2504] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:32:30.475214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1000459-4983-4139-9b23-6af73c817e44 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:30.484397 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1000459-4983-4139-9b23-6af73c817e44 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:30.489107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec34c3-d954-4888-829d-d50a387478c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:30.489754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec34c3-d954-4888-829d-d50a387478c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.490071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec34c3-d954-4888-829d-d50a387478c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.490353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec34c3-d954-4888-829d-d50a387478c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.491058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec34c3-d954-4888-829d-d50a387478c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:30.492962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec34c3-d954-4888-829d-d50a387478c3 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:30.602410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1259/2505] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:32:30 2022] POST /identity/v3/projects => generated 361 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:30.606661 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:30.616539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bee2a-9d3e-47cf-a8b0-b4dac3a05ccd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:30.674947 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efbf2a-820f-45b1-bc34-1f9e63f08588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:30.675624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efbf2a-820f-45b1-bc34-1f9e63f08588 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.675911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efbf2a-820f-45b1-bc34-1f9e63f08588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.676193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efbf2a-820f-45b1-bc34-1f9e63f08588 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.676879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efbf2a-820f-45b1-bc34-1f9e63f08588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:30.678849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efbf2a-820f-45b1-bc34-1f9e63f08588 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:30.766802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1247/2506] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:32:30 2022] POST /identity/v3/users => generated 281 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:30.770861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec34c3-d954-4888-829d-d50a387478c3 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:30.781205 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec34c3-d954-4888-829d-d50a387478c3 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:30.786165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4a76a-bd30-4510-ae00-a31e47eda82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:30.786813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a76a-bd30-4510-ae00-a31e47eda82c admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.787094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a76a-bd30-4510-ae00-a31e47eda82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.787366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a76a-bd30-4510-ae00-a31e47eda82c admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.788158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4a76a-bd30-4510-ae00-a31e47eda82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:30.789737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4a76a-bd30-4510-ae00-a31e47eda82c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:30.800111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1260/2507] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:30.804255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efbf2a-820f-45b1-bc34-1f9e63f08588 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:30.814912 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efbf2a-820f-45b1-bc34-1f9e63f08588 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:30.819593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81a490-4452-4da0-acb6-ee74a11e148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:30.820250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a490-4452-4da0-acb6-ee74a11e148b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.820538 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a490-4452-4da0-acb6-ee74a11e148b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.820817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a490-4452-4da0-acb6-ee74a11e148b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.821541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81a490-4452-4da0-acb6-ee74a11e148b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:30.823345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81a490-4452-4da0-acb6-ee74a11e148b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:30.834187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1248/2508] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:30.838385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4a76a-bd30-4510-ae00-a31e47eda82c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:30.848175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4a76a-bd30-4510-ae00-a31e47eda82c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:30.852770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:30.853640 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.853956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.854236 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] PATH_INFO: `/v3/projects/e07edfa0faeb44f18b0a846ba70dc340/users/7179807443774c4c9fe60d489acae605/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.868208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07edfa0faeb44f18b0a846ba70dc340, user_id=7179807443774c4c9fe60d489acae605, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:30.870930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:30.936926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1261/2509] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:30 2022] PUT /identity/v3/projects/e07edfa0faeb44f18b0a846ba70dc340/users/7179807443774c4c9fe60d489acae605/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:30.948640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f654a-cdfb-40ee-9e29-393111848a8e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:30.948934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f654a-cdfb-40ee-9e29-393111848a8e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:30.949241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f654a-cdfb-40ee-9e29-393111848a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:30.975959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f79f654a-cdfb-40ee-9e29-393111848a8e None None] MFA Rules not processed for user `7179807443774c4c9fe60d489acae605`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:30.979593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f79f654a-cdfb-40ee-9e29-393111848a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:31.038201 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1249/2510] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:32:31.068669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:31.070425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:31.125528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5b7bb-1c68-427f-a311-9291d2e5d1ae admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:31.129134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd84be9-6ba0-41fd-b002-a411c530df5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:31.129777 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd84be9-6ba0-41fd-b002-a411c530df5f service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:31.130090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd84be9-6ba0-41fd-b002-a411c530df5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:31.130372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd84be9-6ba0-41fd-b002-a411c530df5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:31.138432 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd84be9-6ba0-41fd-b002-a411c530df5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:31.139862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd84be9-6ba0-41fd-b002-a411c530df5f service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:31.148853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1262/2511] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:31.162635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f654a-cdfb-40ee-9e29-393111848a8e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:31.223945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f654a-cdfb-40ee-9e29-393111848a8e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:31.228862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:31.229748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:31.230066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:31.230346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] PATH_INFO: `/v3/users/7179807443774c4c9fe60d489acae605` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:31.231602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7179807443774c4c9fe60d489acae605)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:31.233558 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:31.564547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7179807443774c4c9fe60d489acae605'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:31.566258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7179807443774c4c9fe60d489acae605'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:31.566870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7179807443774c4c9fe60d489acae605'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:31.574645 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7179807443774c4c9fe60d489acae605'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:31.674895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7179807443774c4c9fe60d489acae605'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:31.695040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1250/2512] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:32:31 2022] DELETE /identity/v3/users/7179807443774c4c9fe60d489acae605 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:31.699859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6542-3ccc-4dd1-b54a-05b5ba2935c2 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:31.700448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6542-3ccc-4dd1-b54a-05b5ba2935c2 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:31.705575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b6542-3ccc-4dd1-b54a-05b5ba2935c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:31.726904 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-d58b6542-3ccc-4dd1-b54a-05b5ba2935c2 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:31.728119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d58b6542-3ccc-4dd1-b54a-05b5ba2935c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:31.791520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1263/2513] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:31.799638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:31.801093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:31.861595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f01a84-a6e5-44c7-9ca9-0a45672068fa admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:31.866003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772cf1e-45e2-4f68-9e48-9d05d74a56a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:31.866672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772cf1e-45e2-4f68-9e48-9d05d74a56a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:31.866962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772cf1e-45e2-4f68-9e48-9d05d74a56a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:31.867979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772cf1e-45e2-4f68-9e48-9d05d74a56a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:31.883406 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772cf1e-45e2-4f68-9e48-9d05d74a56a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:31.884899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772cf1e-45e2-4f68-9e48-9d05d74a56a1 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:31.895188 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1251/2514] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:32.711440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b6542-3ccc-4dd1-b54a-05b5ba2935c2 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:32.712994 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-d58b6542-3ccc-4dd1-b54a-05b5ba2935c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:32.726371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b6542-3ccc-4dd1-b54a-05b5ba2935c2 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:32.732995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:32.733862 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:32.734650 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:32.734926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] PATH_INFO: `/v3/projects/e07edfa0faeb44f18b0a846ba70dc340` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:32.736767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07edfa0faeb44f18b0a846ba70dc340)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:32.739001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:32.854647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07edfa0faeb44f18b0a846ba70dc340', 'request_id': 'req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:32.859903 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07edfa0faeb44f18b0a846ba70dc340', 'request_id': 'req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:32.956118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07edfa0faeb44f18b0a846ba70dc340', 'request_id': 'req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:32.967753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] The token cache is being invalidate because project e07edfa0faeb44f18b0a846ba70dc340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:32:32.968148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:32.969647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-263c76ee-ca4b-4b20-a6ff-3afe0cb258fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:32.971520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1264/2515] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:32:32 2022] DELETE /identity/v3/projects/e07edfa0faeb44f18b0a846ba70dc340 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:32.978467 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974b1d5-4c38-464b-9aa2-38e2183b6721 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:32.978922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974b1d5-4c38-464b-9aa2-38e2183b6721 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:32.979035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974b1d5-4c38-464b-9aa2-38e2183b6721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:32.996092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1974b1d5-4c38-464b-9aa2-38e2183b6721 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:32.997335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1974b1d5-4c38-464b-9aa2-38e2183b6721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:33.055119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1252/2516] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:33.063331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02c054-84db-4a2f-acf7-836d73d8115d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.063622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02c054-84db-4a2f-acf7-836d73d8115d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.063906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02c054-84db-4a2f-acf7-836d73d8115d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.082926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-4c02c054-84db-4a2f-acf7-836d73d8115d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:33.084123 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4c02c054-84db-4a2f-acf7-836d73d8115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:33.091932 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1265/2517] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:33.096213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974b1d5-4c38-464b-9aa2-38e2183b6721 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:33.106022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974b1d5-4c38-464b-9aa2-38e2183b6721 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:33.110685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:33.111427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.111718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.112001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.112731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:32:33.113040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:33.114675 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:33.126719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1253/2518] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:32:33.130959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02c054-84db-4a2f-acf7-836d73d8115d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:33.140095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02c054-84db-4a2f-acf7-836d73d8115d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:33.145672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b2ed2-9bd7-42bf-9423-66fce030c589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:33.146398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b2ed2-9bd7-42bf-9423-66fce030c589 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.146685 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b2ed2-9bd7-42bf-9423-66fce030c589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.146962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b2ed2-9bd7-42bf-9423-66fce030c589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.147655 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b2ed2-9bd7-42bf-9423-66fce030c589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:33.149616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b2ed2-9bd7-42bf-9423-66fce030c589 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:33.245112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1266/2519] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:32:33 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:33.246294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:33.256576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba4ee1-4ec9-4662-93bb-a73a02802c0d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:33.312611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3af18-f9ba-43dc-8c6e-8a4919120624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:33.313314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3af18-f9ba-43dc-8c6e-8a4919120624 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.313604 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3af18-f9ba-43dc-8c6e-8a4919120624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.313915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3af18-f9ba-43dc-8c6e-8a4919120624 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.314607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3af18-f9ba-43dc-8c6e-8a4919120624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:33.316627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3af18-f9ba-43dc-8c6e-8a4919120624 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:33.459579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1254/2520] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:32:33 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:33.463026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b2ed2-9bd7-42bf-9423-66fce030c589 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:33.473267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b2ed2-9bd7-42bf-9423-66fce030c589 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:33.478069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd23ccf-222d-4842-a9bb-bd34ba4a2e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:33.479034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23ccf-222d-4842-a9bb-bd34ba4a2e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.479034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23ccf-222d-4842-a9bb-bd34ba4a2e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.479305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd23ccf-222d-4842-a9bb-bd34ba4a2e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.480083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd23ccf-222d-4842-a9bb-bd34ba4a2e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:33.481682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd23ccf-222d-4842-a9bb-bd34ba4a2e42 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:33.499505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1267/2521] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:33.500296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3af18-f9ba-43dc-8c6e-8a4919120624 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:33.509562 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3af18-f9ba-43dc-8c6e-8a4919120624 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:33.514525 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3794d0-4cfe-4341-8853-f598b57dd829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:33.515187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3794d0-4cfe-4341-8853-f598b57dd829 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.515473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3794d0-4cfe-4341-8853-f598b57dd829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.515752 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3794d0-4cfe-4341-8853-f598b57dd829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.516516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3794d0-4cfe-4341-8853-f598b57dd829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:33.518156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3794d0-4cfe-4341-8853-f598b57dd829 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:33.528826 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1255/2522] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:33.532811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd23ccf-222d-4842-a9bb-bd34ba4a2e42 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:33.542585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd23ccf-222d-4842-a9bb-bd34ba4a2e42 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:33.547992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbc71b-7d48-4ba1-88ae-fd9399ff7830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:33.548552 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc71b-7d48-4ba1-88ae-fd9399ff7830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.548838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc71b-7d48-4ba1-88ae-fd9399ff7830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.549113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc71b-7d48-4ba1-88ae-fd9399ff7830 admin admin] PATH_INFO: `/v3/projects/50a29871cb8b40e9899135bd6cb5a237/users/772741e96d3e452e86c0736319ac1aa3/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.564333 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbc71b-7d48-4ba1-88ae-fd9399ff7830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a29871cb8b40e9899135bd6cb5a237, user_id=772741e96d3e452e86c0736319ac1aa3, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:33.567581 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbc71b-7d48-4ba1-88ae-fd9399ff7830 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:33.618316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1268/2523] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:33 2022] PUT /identity/v3/projects/50a29871cb8b40e9899135bd6cb5a237/users/772741e96d3e452e86c0736319ac1aa3/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:33.629248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be845a-f309-48eb-9d4c-ab813871c5c8 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.629451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be845a-f309-48eb-9d4c-ab813871c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.629736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be845a-f309-48eb-9d4c-ab813871c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.674336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-e6be845a-f309-48eb-9d4c-ab813871c5c8 None None] MFA Rules not processed for user `772741e96d3e452e86c0736319ac1aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:33.675608 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-e6be845a-f309-48eb-9d4c-ab813871c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:33.770508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1256/2524] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:33.777578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbc71b-7d48-4ba1-88ae-fd9399ff7830 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:33.883868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbc71b-7d48-4ba1-88ae-fd9399ff7830 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:33.888623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2a56c-3ab1-40f7-ad6e-ec1b02709306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:33.889294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2a56c-3ab1-40f7-ad6e-ec1b02709306 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:33.889578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2a56c-3ab1-40f7-ad6e-ec1b02709306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:33.889899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2a56c-3ab1-40f7-ad6e-ec1b02709306 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:33.891481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2a56c-3ab1-40f7-ad6e-ec1b02709306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:33.893490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2a56c-3ab1-40f7-ad6e-ec1b02709306 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:34.009825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1269/2525] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:32:33 2022] POST /identity/v3/projects => generated 373 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:34.011176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be845a-f309-48eb-9d4c-ab813871c5c8 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:34.022034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be845a-f309-48eb-9d4c-ab813871c5c8 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:34.084318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2903-1e44-427f-8315-baa3c1fd3e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:34.085197 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2903-1e44-427f-8315-baa3c1fd3e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:34.085472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2903-1e44-427f-8315-baa3c1fd3e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:34.085751 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2903-1e44-427f-8315-baa3c1fd3e8c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:34.086499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2903-1e44-427f-8315-baa3c1fd3e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:34.088670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2903-1e44-427f-8315-baa3c1fd3e8c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:34.201059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1257/2526] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:32:34 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:34.205490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2a56c-3ab1-40f7-ad6e-ec1b02709306 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:34.216049 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2a56c-3ab1-40f7-ad6e-ec1b02709306 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:34.222043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13903bc9-b60f-4c5e-bffa-b051fffffdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:34.222737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13903bc9-b60f-4c5e-bffa-b051fffffdba admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:34.223017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13903bc9-b60f-4c5e-bffa-b051fffffdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:34.223286 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13903bc9-b60f-4c5e-bffa-b051fffffdba admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:34.223998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13903bc9-b60f-4c5e-bffa-b051fffffdba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:34.225591 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13903bc9-b60f-4c5e-bffa-b051fffffdba admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:34.236462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1270/2527] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:34.240523 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2903-1e44-427f-8315-baa3c1fd3e8c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:34.251069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2903-1e44-427f-8315-baa3c1fd3e8c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:34.255982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e0afa-7020-44d2-b3ef-1731b030e03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:34.256873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e0afa-7020-44d2-b3ef-1731b030e03d admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:34.257183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e0afa-7020-44d2-b3ef-1731b030e03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:34.257460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e0afa-7020-44d2-b3ef-1731b030e03d admin admin] PATH_INFO: `/v3/projects/b04a0d7935a74e42b834bfe72d1bca62/users/1ec3f7a2c560423abe1ae501f7d6804c/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:34.274402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e0afa-7020-44d2-b3ef-1731b030e03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04a0d7935a74e42b834bfe72d1bca62, user_id=1ec3f7a2c560423abe1ae501f7d6804c, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:34.277318 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e0afa-7020-44d2-b3ef-1731b030e03d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:34.384212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1258/2528] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:34 2022] PUT /identity/v3/projects/b04a0d7935a74e42b834bfe72d1bca62/users/1ec3f7a2c560423abe1ae501f7d6804c/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:34.388196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13903bc9-b60f-4c5e-bffa-b051fffffdba admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:34.398659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13903bc9-b60f-4c5e-bffa-b051fffffdba admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:34.452612 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21d876-9337-4106-a34c-e9c49ab1bf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:34.453809 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d876-9337-4106-a34c-e9c49ab1bf2a admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:34.454105 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d876-9337-4106-a34c-e9c49ab1bf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:34.454427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d876-9337-4106-a34c-e9c49ab1bf2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:34.455151 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21d876-9337-4106-a34c-e9c49ab1bf2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:34.456855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21d876-9337-4106-a34c-e9c49ab1bf2a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:34.466895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1271/2529] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:34.471735 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e0afa-7020-44d2-b3ef-1731b030e03d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:34.486985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e0afa-7020-44d2-b3ef-1731b030e03d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:34.491840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:34.492726 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:34.493028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:34.493305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] PATH_INFO: `/v3/projects/b04a0d7935a74e42b834bfe72d1bca62/users/1ec3f7a2c560423abe1ae501f7d6804c/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:34.498766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04a0d7935a74e42b834bfe72d1bca62, user_id=1ec3f7a2c560423abe1ae501f7d6804c, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:34.502394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:34.513733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1259/2530] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:34 2022] PUT /identity/v3/projects/b04a0d7935a74e42b834bfe72d1bca62/users/1ec3f7a2c560423abe1ae501f7d6804c/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:34.525494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a168b-7f66-4ae8-aca7-7efb680f1941 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:34.525818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a168b-7f66-4ae8-aca7-7efb680f1941 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:34.526121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a168b-7f66-4ae8-aca7-7efb680f1941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:34.554344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-020a168b-7f66-4ae8-aca7-7efb680f1941 None None] MFA Rules not processed for user `1ec3f7a2c560423abe1ae501f7d6804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:34.555567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-020a168b-7f66-4ae8-aca7-7efb680f1941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:34.622409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1272/2531] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:34.633676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:34.635449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:34.691711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772496d7-7297-4c17-bb22-41ef7c209f06 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:34.695555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac043139-0302-4c16-b9ad-f1d440a30b22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:34.696237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac043139-0302-4c16-b9ad-f1d440a30b22 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:34.696520 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac043139-0302-4c16-b9ad-f1d440a30b22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:34.696801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac043139-0302-4c16-b9ad-f1d440a30b22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:34.754225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac043139-0302-4c16-b9ad-f1d440a30b22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:34.755781 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac043139-0302-4c16-b9ad-f1d440a30b22 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:34.764705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1260/2532] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:37.549990 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a168b-7f66-4ae8-aca7-7efb680f1941 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:37.551606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-020a168b-7f66-4ae8-aca7-7efb680f1941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:37.612661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a168b-7f66-4ae8-aca7-7efb680f1941 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:37.616558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b08b1-f935-4377-9b1d-737ce9436526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:37.617388 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b08b1-f935-4377-9b1d-737ce9436526 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:37.617674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b08b1-f935-4377-9b1d-737ce9436526 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:37.619164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b08b1-f935-4377-9b1d-737ce9436526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:37.621116 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9e3b08b1-f935-4377-9b1d-737ce9436526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:37.684995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b08b1-f935-4377-9b1d-737ce9436526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:37.686807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b08b1-f935-4377-9b1d-737ce9436526 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:37.697675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1273/2533] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:37.874615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac043139-0302-4c16-b9ad-f1d440a30b22 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:37.944749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac043139-0302-4c16-b9ad-f1d440a30b22 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:37.949468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:37.950410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:37.950694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:37.950973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] PATH_INFO: `/v3/users/772741e96d3e452e86c0736319ac1aa3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:37.952224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=772741e96d3e452e86c0736319ac1aa3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:37.955435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:38.135527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '772741e96d3e452e86c0736319ac1aa3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:38.137849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '772741e96d3e452e86c0736319ac1aa3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:38.139103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '772741e96d3e452e86c0736319ac1aa3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:38.148946 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '772741e96d3e452e86c0736319ac1aa3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:38.212844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '772741e96d3e452e86c0736319ac1aa3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:38.243620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1261/2534] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:32:37 2022] DELETE /identity/v3/users/772741e96d3e452e86c0736319ac1aa3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:38.246196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25546c-8922-4ee4-b2f7-5b4620db337b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:38.246540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25546c-8922-4ee4-b2f7-5b4620db337b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:38.246851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25546c-8922-4ee4-b2f7-5b4620db337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:38.271772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ce25546c-8922-4ee4-b2f7-5b4620db337b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:38.273150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ce25546c-8922-4ee4-b2f7-5b4620db337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:38.355890 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1274/2535] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:38.369628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:38.371687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:38.455370 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8463d7b-3e32-461f-8e68-8448846cca94 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:38.459879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f39eb-3aa9-4d3e-997d-cbb0aa98c39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:38.460623 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f39eb-3aa9-4d3e-997d-cbb0aa98c39e service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:38.460976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f39eb-3aa9-4d3e-997d-cbb0aa98c39e service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:38.461369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f39eb-3aa9-4d3e-997d-cbb0aa98c39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:38.477125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f39eb-3aa9-4d3e-997d-cbb0aa98c39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:38.477125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f39eb-3aa9-4d3e-997d-cbb0aa98c39e service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:38.488363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1262/2536] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:39.146301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25546c-8922-4ee4-b2f7-5b4620db337b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:39.148386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ce25546c-8922-4ee4-b2f7-5b4620db337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:39.160802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25546c-8922-4ee4-b2f7-5b4620db337b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:39.165728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:39.166767 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:39.167155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:39.167694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] PATH_INFO: `/v3/projects/50a29871cb8b40e9899135bd6cb5a237` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:39.169271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a29871cb8b40e9899135bd6cb5a237)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:39.171398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:39.334621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a29871cb8b40e9899135bd6cb5a237', 'request_id': 'req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.340146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a29871cb8b40e9899135bd6cb5a237', 'request_id': 'req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.442091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a29871cb8b40e9899135bd6cb5a237', 'request_id': 'req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.459770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] The token cache is being invalidate because project 50a29871cb8b40e9899135bd6cb5a237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:32:39.460066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.460984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.465836 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1275/2537] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:32:39 2022] DELETE /identity/v3/projects/50a29871cb8b40e9899135bd6cb5a237 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:39.468286 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f39eb-3aa9-4d3e-997d-cbb0aa98c39e service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:39.486639 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f39eb-3aa9-4d3e-997d-cbb0aa98c39e service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:39.585101 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:39.585923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:39.586205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:39.586496 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] PATH_INFO: `/v3/users/1ec3f7a2c560423abe1ae501f7d6804c` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:39.588047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ec3f7a2c560423abe1ae501f7d6804c)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:39.591398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:39.779590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec3f7a2c560423abe1ae501f7d6804c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.780274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec3f7a2c560423abe1ae501f7d6804c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.781405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec3f7a2c560423abe1ae501f7d6804c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.788407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec3f7a2c560423abe1ae501f7d6804c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.855032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7c4d0357-d27c-48bc-a6ff-bad320cf2477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec3f7a2c560423abe1ae501f7d6804c'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:39.874804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1263/2538] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:32:39 2022] DELETE /identity/v3/users/1ec3f7a2c560423abe1ae501f7d6804c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:40.385963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:40.385963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:40.428604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba1a93-57f0-480f-9764-c65cfd1a8c8b admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:40.516373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:40.517558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:40.518078 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:40.518382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] PATH_INFO: `/v3/projects/b04a0d7935a74e42b834bfe72d1bca62` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:40.519876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04a0d7935a74e42b834bfe72d1bca62)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:40.522454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:40.678071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04a0d7935a74e42b834bfe72d1bca62', 'request_id': 'req-50357504-fc5a-4da3-9cdd-eb7b90804ca9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:40.684431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04a0d7935a74e42b834bfe72d1bca62', 'request_id': 'req-50357504-fc5a-4da3-9cdd-eb7b90804ca9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:40.744133 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04a0d7935a74e42b834bfe72d1bca62', 'request_id': 'req-50357504-fc5a-4da3-9cdd-eb7b90804ca9'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:40.769082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] The token cache is being invalidate because project b04a0d7935a74e42b834bfe72d1bca62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:32:40.769644 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:40.770802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-50357504-fc5a-4da3-9cdd-eb7b90804ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:40.783911 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1276/2539] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:32:40 2022] DELETE /identity/v3/projects/b04a0d7935a74e42b834bfe72d1bca62 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:40.789022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c33d-b1e9-4ee1-9b16-f8d0209b1b99 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:40.789596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c33d-b1e9-4ee1-9b16-f8d0209b1b99 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:40.790212 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c33d-b1e9-4ee1-9b16-f8d0209b1b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:40.821210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b1a1c33d-b1e9-4ee1-9b16-f8d0209b1b99 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:40.823087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b1a1c33d-b1e9-4ee1-9b16-f8d0209b1b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:40.909611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1aa715-3589-4b8b-a6b1-b6d23eb72589 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:40.909891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1aa715-3589-4b8b-a6b1-b6d23eb72589 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:40.910434 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1aa715-3589-4b8b-a6b1-b6d23eb72589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:40.913243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1264/2540] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:40.933160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0d1aa715-3589-4b8b-a6b1-b6d23eb72589 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:40.934339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0d1aa715-3589-4b8b-a6b1-b6d23eb72589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:40.943907 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1277/2541] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:32:40.950621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1c33d-b1e9-4ee1-9b16-f8d0209b1b99 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:40.964896 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1c33d-b1e9-4ee1-9b16-f8d0209b1b99 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:40.972974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:40.973813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:40.974204 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:40.974557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:40.975380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:32:40.975885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:40.977636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:40.992607 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1265/2542] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:32:40.997198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1aa715-3589-4b8b-a6b1-b6d23eb72589 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:41.011171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1aa715-3589-4b8b-a6b1-b6d23eb72589 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:41.016322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787b316-24f3-4ccb-9f8b-b7755ef7bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:41.016986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787b316-24f3-4ccb-9f8b-b7755ef7bbda admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:41.017413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787b316-24f3-4ccb-9f8b-b7755ef7bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:41.017739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787b316-24f3-4ccb-9f8b-b7755ef7bbda admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:41.018669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787b316-24f3-4ccb-9f8b-b7755ef7bbda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:41.021406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787b316-24f3-4ccb-9f8b-b7755ef7bbda admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:41.132972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1278/2543] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:32:40 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:32:41.138839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:41.152157 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691309db-af75-43b4-9008-e359f127c027 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:41.232344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb24e-2428-4e92-92d8-7ad436f541e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:41.233028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb24e-2428-4e92-92d8-7ad436f541e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:41.233314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb24e-2428-4e92-92d8-7ad436f541e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:41.233601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb24e-2428-4e92-92d8-7ad436f541e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:41.235003 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb24e-2428-4e92-92d8-7ad436f541e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:41.237299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb24e-2428-4e92-92d8-7ad436f541e3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:41.366062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1266/2544] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:32:41 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:32:41.367180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787b316-24f3-4ccb-9f8b-b7755ef7bbda admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:41.380053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787b316-24f3-4ccb-9f8b-b7755ef7bbda admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:41.384922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c73ea5-310c-4a6d-b290-ec20cd349fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:41.385604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c73ea5-310c-4a6d-b290-ec20cd349fde admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:41.385963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c73ea5-310c-4a6d-b290-ec20cd349fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:41.386240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c73ea5-310c-4a6d-b290-ec20cd349fde admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:41.387019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c73ea5-310c-4a6d-b290-ec20cd349fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:41.388617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c73ea5-310c-4a6d-b290-ec20cd349fde admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:41.400263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1279/2545] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:32:41.405136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb24e-2428-4e92-92d8-7ad436f541e3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:41.418158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb24e-2428-4e92-92d8-7ad436f541e3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:41.423539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fa4e5-d96d-4c34-ae3d-c24839dd1b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:41.426355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa4e5-d96d-4c34-ae3d-c24839dd1b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:41.426355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa4e5-d96d-4c34-ae3d-c24839dd1b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:41.426355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa4e5-d96d-4c34-ae3d-c24839dd1b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:41.426355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fa4e5-d96d-4c34-ae3d-c24839dd1b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:41.427855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fa4e5-d96d-4c34-ae3d-c24839dd1b9f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:41.441185 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1267/2546] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:41.446741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c73ea5-310c-4a6d-b290-ec20cd349fde admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:41.461870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c73ea5-310c-4a6d-b290-ec20cd349fde admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:41.469196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:41.470609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:41.471113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:41.471504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] PATH_INFO: `/v3/projects/3ceb8d72199c41d09b49b7eabc289039/users/35babdbdaac942ba845fd289eda01e46/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:41.502927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ceb8d72199c41d09b49b7eabc289039, user_id=35babdbdaac942ba845fd289eda01e46, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:41.506771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:41.602800 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1280/2547] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:41 2022] PUT /identity/v3/projects/3ceb8d72199c41d09b49b7eabc289039/users/35babdbdaac942ba845fd289eda01e46/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:41.614429 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c7f82-036d-4b2b-ada6-683648474d56 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:41.614727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c7f82-036d-4b2b-ada6-683648474d56 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:41.615012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c7f82-036d-4b2b-ada6-683648474d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:41.642515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ca6c7f82-036d-4b2b-ada6-683648474d56 None None] MFA Rules not processed for user `35babdbdaac942ba845fd289eda01e46`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:41.643899 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ca6c7f82-036d-4b2b-ada6-683648474d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:41.708067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1268/2548] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:41.719900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:41.721715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:41.777257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab94e-5f9c-458f-8422-f1a1b35b1bfc admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:41.781491 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9dff9c-95d1-4536-8d4a-7af7577f6cdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:41.782247 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dff9c-95d1-4536-8d4a-7af7577f6cdb service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:41.782527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dff9c-95d1-4536-8d4a-7af7577f6cdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:41.782796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dff9c-95d1-4536-8d4a-7af7577f6cdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:41.791747 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9dff9c-95d1-4536-8d4a-7af7577f6cdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:41.793379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9dff9c-95d1-4536-8d4a-7af7577f6cdb service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:41.802867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1281/2549] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:44.994405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c7f82-036d-4b2b-ada6-683648474d56 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:45.054693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c7f82-036d-4b2b-ada6-683648474d56 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:45.059224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:45.060086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:45.060373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:45.060651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] PATH_INFO: `/v3/users/35babdbdaac942ba845fd289eda01e46` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:45.062354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=35babdbdaac942ba845fd289eda01e46)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:45.064402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:45.442616 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35babdbdaac942ba845fd289eda01e46'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:45.443227 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35babdbdaac942ba845fd289eda01e46'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:45.443823 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35babdbdaac942ba845fd289eda01e46'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:45.450339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35babdbdaac942ba845fd289eda01e46'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:45.529540 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35babdbdaac942ba845fd289eda01e46'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:45.553583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1269/2550] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:32:44 2022] DELETE /identity/v3/users/35babdbdaac942ba845fd289eda01e46 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:45.558190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6e0e-6ac1-438d-a76c-a35b73d12953 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:45.558807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6e0e-6ac1-438d-a76c-a35b73d12953 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:45.559128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e6e0e-6ac1-438d-a76c-a35b73d12953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:45.577365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-f51e6e0e-6ac1-438d-a76c-a35b73d12953 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:45.578661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f51e6e0e-6ac1-438d-a76c-a35b73d12953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:45.631418 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:45.633143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:45.642385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1282/2551] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:45.651854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e6e0e-6ac1-438d-a76c-a35b73d12953 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:45.653220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-f51e6e0e-6ac1-438d-a76c-a35b73d12953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:45.691742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43675862-7bb3-4158-babc-07f4193c2e0e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:45.695363 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:45.696059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:45.696348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:45.696628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:45.698652 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:45.711448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:45.712066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e6e0e-6ac1-438d-a76c-a35b73d12953 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:45.712885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:45.715664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5532b-acf9-4fbc-ac0f-e61d3974e5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:45.716448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5532b-acf9-4fbc-ac0f-e61d3974e5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:45.716728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5532b-acf9-4fbc-ac0f-e61d3974e5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:45.717001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5532b-acf9-4fbc-ac0f-e61d3974e5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:45.723132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1270/2552] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:32:45.727159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5532b-acf9-4fbc-ac0f-e61d3974e5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:45.728897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5532b-acf9-4fbc-ac0f-e61d3974e5e8 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:45.739481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1283/2553] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:32:46.346360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:46.348744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:46.364247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3d6ea-bfcd-49eb-a363-46198ed246d5 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:46.370642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:46.371647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:46.372046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:46.372220 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] PATH_INFO: `/v3/projects/3ceb8d72199c41d09b49b7eabc289039` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:46.373681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ceb8d72199c41d09b49b7eabc289039)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:46.380439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:46.487257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ceb8d72199c41d09b49b7eabc289039', 'request_id': 'req-0782e455-5568-4a0d-b074-006dff23e599'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:46.493118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ceb8d72199c41d09b49b7eabc289039', 'request_id': 'req-0782e455-5568-4a0d-b074-006dff23e599'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:46.578402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ceb8d72199c41d09b49b7eabc289039', 'request_id': 'req-0782e455-5568-4a0d-b074-006dff23e599'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:46.591938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] The token cache is being invalidate because project 3ceb8d72199c41d09b49b7eabc289039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:32:46.592288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:46.592995 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0782e455-5568-4a0d-b074-006dff23e599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:46.594603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1271/2554] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:32:46 2022] DELETE /identity/v3/projects/3ceb8d72199c41d09b49b7eabc289039 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:46.599504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c81dbc-c044-4f1a-9f13-f8069403df04 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:46.599987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c81dbc-c044-4f1a-9f13-f8069403df04 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:46.600103 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c81dbc-c044-4f1a-9f13-f8069403df04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:46.619418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-14c81dbc-c044-4f1a-9f13-f8069403df04 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:46.620912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-14c81dbc-c044-4f1a-9f13-f8069403df04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:46.685821 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1284/2555] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:32:46.694435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab4d3-19a9-4646-87e2-7995e15e583c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:46.694728 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab4d3-19a9-4646-87e2-7995e15e583c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:46.695012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab4d3-19a9-4646-87e2-7995e15e583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:46.714252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5f0ab4d3-19a9-4646-87e2-7995e15e583c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:46.715504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5f0ab4d3-19a9-4646-87e2-7995e15e583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:46.724235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1272/2556] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:46.728609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c81dbc-c044-4f1a-9f13-f8069403df04 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:46.738066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c81dbc-c044-4f1a-9f13-f8069403df04 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:46.742628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:46.743289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:46.743569 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:46.743841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:46.744565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:32:46.744873 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:46.746514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:46.759429 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1285/2557] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:32:46.763413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ab4d3-19a9-4646-87e2-7995e15e583c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:46.773866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ab4d3-19a9-4646-87e2-7995e15e583c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:46.779221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e584a-bc8a-4b97-925c-59687d85bac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:46.779884 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e584a-bc8a-4b97-925c-59687d85bac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:46.780167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e584a-bc8a-4b97-925c-59687d85bac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:46.780438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e584a-bc8a-4b97-925c-59687d85bac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:46.781198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e584a-bc8a-4b97-925c-59687d85bac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:46.783148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e584a-bc8a-4b97-925c-59687d85bac7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:46.888061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1273/2558] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:32:46 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:46.894175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:46.919828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48486d9c-b6f4-4714-b3ba-e6c85b67ff2d admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:46.982454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2430501-942c-46bc-a19e-2c7e06c3e4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:46.983301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2430501-942c-46bc-a19e-2c7e06c3e4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:46.984422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2430501-942c-46bc-a19e-2c7e06c3e4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:46.984781 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2430501-942c-46bc-a19e-2c7e06c3e4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:46.985722 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2430501-942c-46bc-a19e-2c7e06c3e4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:46.987976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2430501-942c-46bc-a19e-2c7e06c3e4f1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:47.064739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1286/2559] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:32:46 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:47.069032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e584a-bc8a-4b97-925c-59687d85bac7 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:47.080118 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e584a-bc8a-4b97-925c-59687d85bac7 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:47.085237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608185a-d4ed-4225-8271-2da10531422f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:47.085981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608185a-d4ed-4225-8271-2da10531422f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:47.086267 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608185a-d4ed-4225-8271-2da10531422f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:47.086548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608185a-d4ed-4225-8271-2da10531422f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:47.087294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608185a-d4ed-4225-8271-2da10531422f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:47.088936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608185a-d4ed-4225-8271-2da10531422f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:47.100393 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1274/2560] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:47.104437 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2430501-942c-46bc-a19e-2c7e06c3e4f1 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:47.115737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2430501-942c-46bc-a19e-2c7e06c3e4f1 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:47.120860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d1836-05b7-471f-aa3e-5569672d84bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:47.121558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d1836-05b7-471f-aa3e-5569672d84bc admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:47.121881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d1836-05b7-471f-aa3e-5569672d84bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:47.122166 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d1836-05b7-471f-aa3e-5569672d84bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:47.122906 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d1836-05b7-471f-aa3e-5569672d84bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:47.125118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d1836-05b7-471f-aa3e-5569672d84bc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:47.135734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1287/2561] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:47.140136 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608185a-d4ed-4225-8271-2da10531422f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:47.150587 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608185a-d4ed-4225-8271-2da10531422f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:47.155337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:47.156235 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:47.156518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:47.156794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] PATH_INFO: `/v3/projects/0ebe87e3f2c54a34aa824ce542d24cfa/users/3ea3e6b8869448d9a9bc40f985b90de1/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:47.172530 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebe87e3f2c54a34aa824ce542d24cfa, user_id=3ea3e6b8869448d9a9bc40f985b90de1, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:47.175249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:47.270965 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1275/2562] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:47 2022] PUT /identity/v3/projects/0ebe87e3f2c54a34aa824ce542d24cfa/users/3ea3e6b8869448d9a9bc40f985b90de1/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:32:47.282679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3064-3301-419c-ba9b-1d6533566eeb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:47.282679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3064-3301-419c-ba9b-1d6533566eeb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:47.282679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3064-3301-419c-ba9b-1d6533566eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:47.327256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-a1fe3064-3301-419c-ba9b-1d6533566eeb None None] MFA Rules not processed for user `3ea3e6b8869448d9a9bc40f985b90de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:47.328550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a1fe3064-3301-419c-ba9b-1d6533566eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:47.409381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1288/2563] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:47.424847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:47.426722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:47.486183 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c03c-23d1-49c6-8f91-897dc69d9c77 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:47.489953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7ab5d-2b85-4bb9-bb05-0e148cc3e3a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3f786b791b4c8796d251948917ca93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:47.490628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ab5d-2b85-4bb9-bb05-0e148cc3e3a2 service glance] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:47.490988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ab5d-2b85-4bb9-bb05-0e148cc3e3a2 service glance] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:47.491313 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ab5d-2b85-4bb9-bb05-0e148cc3e3a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:47.500444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7ab5d-2b85-4bb9-bb05-0e148cc3e3a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:47.501982 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7ab5d-2b85-4bb9-bb05-0e148cc3e3a2 service glance] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:47.511240 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1276/2564] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:32:49.847975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe3064-3301-419c-ba9b-1d6533566eeb None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:49.849567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a1fe3064-3301-419c-ba9b-1d6533566eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:49.929804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe3064-3301-419c-ba9b-1d6533566eeb None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:49.933849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:49.934734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:49.935018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:49.935295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] PATH_INFO: `/v3/users/08b9fbf87cfa417fbea857ae602a140e` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:49.936492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08b9fbf87cfa417fbea857ae602a140e)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:49.939301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:50.191936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08b9fbf87cfa417fbea857ae602a140e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.192530 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08b9fbf87cfa417fbea857ae602a140e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.274842 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08b9fbf87cfa417fbea857ae602a140e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.282443 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08b9fbf87cfa417fbea857ae602a140e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.291600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-d8c5cfec-cb43-4d03-b696-34b740b7f995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08b9fbf87cfa417fbea857ae602a140e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.300981 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1289/2565] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:32:49 2022] DELETE /identity/v3/users/08b9fbf87cfa417fbea857ae602a140e => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:50.608256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7ab5d-2b85-4bb9-bb05-0e148cc3e3a2 service glance] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:50.621807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7ab5d-2b85-4bb9-bb05-0e148cc3e3a2 service glance] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:50.684382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:50.685253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:50.685535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:50.685838 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] PATH_INFO: `/v3/projects/4cd4aa315c4340a1a649d8c51cea0da9` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:50.687107 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd4aa315c4340a1a649d8c51cea0da9)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:50.689036 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:50.813258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd4aa315c4340a1a649d8c51cea0da9', 'request_id': 'req-eb379808-4607-42b3-83be-f1fbbfe1c032'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.818947 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd4aa315c4340a1a649d8c51cea0da9', 'request_id': 'req-eb379808-4607-42b3-83be-f1fbbfe1c032'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.880263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd4aa315c4340a1a649d8c51cea0da9', 'request_id': 'req-eb379808-4607-42b3-83be-f1fbbfe1c032'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.894499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] The token cache is being invalidate because project 4cd4aa315c4340a1a649d8c51cea0da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:32:50.894849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.895573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-eb379808-4607-42b3-83be-f1fbbfe1c032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:32:50.897516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1277/2566] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:32:50 2022] DELETE /identity/v3/projects/4cd4aa315c4340a1a649d8c51cea0da9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:50.903069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80c5df-7368-417e-909d-62dd66fabb1e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:50.903678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80c5df-7368-417e-909d-62dd66fabb1e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:50.903678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80c5df-7368-417e-909d-62dd66fabb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:50.922306 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1e80c5df-7368-417e-909d-62dd66fabb1e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:50.926241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1e80c5df-7368-417e-909d-62dd66fabb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:50.986356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1290/2567] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:50.994374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c543c-5535-4b54-845c-2cdb8c48c51f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:50.994664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c543c-5535-4b54-845c-2cdb8c48c51f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:50.994967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c543c-5535-4b54-845c-2cdb8c48c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.017200 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4c0c543c-5535-4b54-845c-2cdb8c48c51f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:51.018606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4c0c543c-5535-4b54-845c-2cdb8c48c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:51.026566 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1278/2568] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:32:51.031378 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80c5df-7368-417e-909d-62dd66fabb1e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:51.041195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80c5df-7368-417e-909d-62dd66fabb1e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:51.046263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:51.047017 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:51.047306 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:51.047590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.048340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:32:51.048652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:51.050353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:51.063438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1291/2569] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:32:51.068369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c543c-5535-4b54-845c-2cdb8c48c51f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:51.079050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c543c-5535-4b54-845c-2cdb8c48c51f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:51.084067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f592e4e-f523-4002-a6c8-7501e17a7cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:51.084743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f592e4e-f523-4002-a6c8-7501e17a7cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:51.085032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f592e4e-f523-4002-a6c8-7501e17a7cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:51.085307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f592e4e-f523-4002-a6c8-7501e17a7cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.086039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f592e4e-f523-4002-a6c8-7501e17a7cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:51.087966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f592e4e-f523-4002-a6c8-7501e17a7cb1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:51.184795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1279/2570] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:32:51 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:32:51.206551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:51.217058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79c913-b4bf-4e0b-8735-e22f8ac6314e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:51.271386 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8741c2-c82d-41ce-8d15-8d869d28b17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:51.272101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8741c2-c82d-41ce-8d15-8d869d28b17e admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:51.272474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8741c2-c82d-41ce-8d15-8d869d28b17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:51.272803 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8741c2-c82d-41ce-8d15-8d869d28b17e admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.273516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8741c2-c82d-41ce-8d15-8d869d28b17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:51.275492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8741c2-c82d-41ce-8d15-8d869d28b17e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:51.395197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1292/2571] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:32:51 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:32:51.400536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f592e4e-f523-4002-a6c8-7501e17a7cb1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:51.410856 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f592e4e-f523-4002-a6c8-7501e17a7cb1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:51.415307 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a93b70-cc3d-4ac9-868c-40ef204aac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:51.415953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93b70-cc3d-4ac9-868c-40ef204aac46 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:51.416233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93b70-cc3d-4ac9-868c-40ef204aac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:51.416507 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93b70-cc3d-4ac9-868c-40ef204aac46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.417219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a93b70-cc3d-4ac9-868c-40ef204aac46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:51.418847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a93b70-cc3d-4ac9-868c-40ef204aac46 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:51.430260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1280/2572] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:51.434373 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8741c2-c82d-41ce-8d15-8d869d28b17e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:51.445363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8741c2-c82d-41ce-8d15-8d869d28b17e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:51.450161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053e0b2-8a1e-46b4-893b-096948bae5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:51.450825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053e0b2-8a1e-46b4-893b-096948bae5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:51.451107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053e0b2-8a1e-46b4-893b-096948bae5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:51.451380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053e0b2-8a1e-46b4-893b-096948bae5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.452118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053e0b2-8a1e-46b4-893b-096948bae5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:51.453713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053e0b2-8a1e-46b4-893b-096948bae5e8 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:51.464318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1293/2573] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:32:51.468693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a93b70-cc3d-4ac9-868c-40ef204aac46 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:51.479020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a93b70-cc3d-4ac9-868c-40ef204aac46 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:51.484129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:51.485019 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:51.485305 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:51.485583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] PATH_INFO: `/v3/projects/ec6436a7141248d58e05e8f156cfda29/users/50c3ffe59ef44c11a3cf19648b81287c/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.501674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6436a7141248d58e05e8f156cfda29, user_id=50c3ffe59ef44c11a3cf19648b81287c, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:51.504448 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:51.590073 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1281/2574] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:32:51 2022] PUT /identity/v3/projects/ec6436a7141248d58e05e8f156cfda29/users/50c3ffe59ef44c11a3cf19648b81287c/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:32:51.600665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5061b5c-fa21-4702-9d9e-f8ca802896c7 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:51.602657 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5061b5c-fa21-4702-9d9e-f8ca802896c7 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:51.602657 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5061b5c-fa21-4702-9d9e-f8ca802896c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.634218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-b5061b5c-fa21-4702-9d9e-f8ca802896c7 None None] MFA Rules not processed for user `50c3ffe59ef44c11a3cf19648b81287c`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:32:51.635446 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-b5061b5c-fa21-4702-9d9e-f8ca802896c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:51.700692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1294/2575] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:32:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:32:51.712128 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:32:51.713842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:32:51.787063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99329dd-ea6a-4b6b-b7fa-1297b82cb303 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:32:51.792634 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edc61f-6a84-4c91-be7e-658163293a46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:32:51.793282 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edc61f-6a84-4c91-be7e-658163293a46 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:32:51.793567 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edc61f-6a84-4c91-be7e-658163293a46 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:32:51.793878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edc61f-6a84-4c91-be7e-658163293a46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:32:51.803328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edc61f-6a84-4c91-be7e-658163293a46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:32:51.804939 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edc61f-6a84-4c91-be7e-658163293a46 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:32:51.814054 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1282/2576] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:32:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:00.944492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5061b5c-fa21-4702-9d9e-f8ca802896c7 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:01.010744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5061b5c-fa21-4702-9d9e-f8ca802896c7 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:01.015804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:01.016677 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:01.016966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:01.017243 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] PATH_INFO: `/v3/users/50c3ffe59ef44c11a3cf19648b81287c` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:01.018522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c3ffe59ef44c11a3cf19648b81287c)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:01.020453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:01.190541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c3ffe59ef44c11a3cf19648b81287c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:01.191193 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c3ffe59ef44c11a3cf19648b81287c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:01.280069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c3ffe59ef44c11a3cf19648b81287c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:01.287346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c3ffe59ef44c11a3cf19648b81287c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:01.288047 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c3ffe59ef44c11a3cf19648b81287c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:01.305084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1295/2577] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:00 2022] DELETE /identity/v3/users/50c3ffe59ef44c11a3cf19648b81287c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:01.309808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ae88-16ed-4e57-9922-f41f72c714c7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:01.310272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ae88-16ed-4e57-9922-f41f72c714c7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:01.310386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ae88-16ed-4e57-9922-f41f72c714c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:01.327335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8105ae88-16ed-4e57-9922-f41f72c714c7 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:01.329203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8105ae88-16ed-4e57-9922-f41f72c714c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:01.385472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1283/2578] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:01.395784 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:01.397264 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:01.451987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b7ae7-34c8-4260-b5f1-ed9f183949b7 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:01.456235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ec82b-a9c7-4ba5-bf82-deb43fe3198d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:01.456870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ec82b-a9c7-4ba5-bf82-deb43fe3198d service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:01.457150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ec82b-a9c7-4ba5-bf82-deb43fe3198d service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:01.457425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ec82b-a9c7-4ba5-bf82-deb43fe3198d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:01.466385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ec82b-a9c7-4ba5-bf82-deb43fe3198d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:01.467802 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ec82b-a9c7-4ba5-bf82-deb43fe3198d service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:01.477129 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1296/2579] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:01.894270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105ae88-16ed-4e57-9922-f41f72c714c7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:01.895845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8105ae88-16ed-4e57-9922-f41f72c714c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:01.912081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105ae88-16ed-4e57-9922-f41f72c714c7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:01.917417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:01.918324 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:01.918605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:01.918881 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] PATH_INFO: `/v3/projects/ec6436a7141248d58e05e8f156cfda29` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:01.920242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6436a7141248d58e05e8f156cfda29)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:01.922179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:02.028502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6436a7141248d58e05e8f156cfda29', 'request_id': 'req-7b6796f5-15cc-481a-a072-1b8f65559201'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:02.034134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6436a7141248d58e05e8f156cfda29', 'request_id': 'req-7b6796f5-15cc-481a-a072-1b8f65559201'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:02.109065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6436a7141248d58e05e8f156cfda29', 'request_id': 'req-7b6796f5-15cc-481a-a072-1b8f65559201'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:02.120492 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] The token cache is being invalidate because project ec6436a7141248d58e05e8f156cfda29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:02.120781 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:02.121402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-7b6796f5-15cc-481a-a072-1b8f65559201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:02.123000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1284/2580] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:01 2022] DELETE /identity/v3/projects/ec6436a7141248d58e05e8f156cfda29 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:02.131698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65915829-ca76-414f-bc88-75e7fdc27d5f None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.132175 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65915829-ca76-414f-bc88-75e7fdc27d5f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.132282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65915829-ca76-414f-bc88-75e7fdc27d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.150611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-65915829-ca76-414f-bc88-75e7fdc27d5f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:02.151820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-65915829-ca76-414f-bc88-75e7fdc27d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:02.210323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1297/2581] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:02.217860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc892d6-208e-4db7-9109-ab6189202e8b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.218156 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc892d6-208e-4db7-9109-ab6189202e8b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.218568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc892d6-208e-4db7-9109-ab6189202e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.236521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-1cc892d6-208e-4db7-9109-ab6189202e8b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:02.237731 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1cc892d6-208e-4db7-9109-ab6189202e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:02.245613 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1285/2582] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:02.250111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65915829-ca76-414f-bc88-75e7fdc27d5f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:02.259545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65915829-ca76-414f-bc88-75e7fdc27d5f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:02.264289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:02.265031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.265879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.266259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.267038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:33:02.267352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:02.269100 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:02.280287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1298/2583] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:33:02.284332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc892d6-208e-4db7-9109-ab6189202e8b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:02.293232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc892d6-208e-4db7-9109-ab6189202e8b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:02.298427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8360e-842e-4ee2-bb53-f10cfa1bf6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:02.299088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8360e-842e-4ee2-bb53-f10cfa1bf6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.299417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8360e-842e-4ee2-bb53-f10cfa1bf6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.299706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8360e-842e-4ee2-bb53-f10cfa1bf6b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.300410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8360e-842e-4ee2-bb53-f10cfa1bf6b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:02.302371 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8360e-842e-4ee2-bb53-f10cfa1bf6b5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:02.391112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1286/2584] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:02 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:02.395323 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:02.405566 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984f7bf-ad13-42ee-b247-2c8bf575f5b4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:02.459465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bea596-47c3-4965-9dd3-e5dc60c37a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:02.460125 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bea596-47c3-4965-9dd3-e5dc60c37a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.460461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bea596-47c3-4965-9dd3-e5dc60c37a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.460682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bea596-47c3-4965-9dd3-e5dc60c37a5c admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.461357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bea596-47c3-4965-9dd3-e5dc60c37a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:02.463278 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bea596-47c3-4965-9dd3-e5dc60c37a5c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:02.560860 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1299/2585] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:02 2022] POST /identity/v3/users => generated 302 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:02.565468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8360e-842e-4ee2-bb53-f10cfa1bf6b5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:02.575651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8360e-842e-4ee2-bb53-f10cfa1bf6b5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:02.580219 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bc304-1c32-4f6d-aa4e-759023a158a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:02.580849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bc304-1c32-4f6d-aa4e-759023a158a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.581130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bc304-1c32-4f6d-aa4e-759023a158a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.581411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bc304-1c32-4f6d-aa4e-759023a158a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.582153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0bc304-1c32-4f6d-aa4e-759023a158a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:02.583994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0bc304-1c32-4f6d-aa4e-759023a158a2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:02.594012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1287/2586] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:02.597908 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bea596-47c3-4965-9dd3-e5dc60c37a5c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:02.607729 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bea596-47c3-4965-9dd3-e5dc60c37a5c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:02.612850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cda9f-f2d3-4df6-8caa-b843643a8920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:02.613501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cda9f-f2d3-4df6-8caa-b843643a8920 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.613808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cda9f-f2d3-4df6-8caa-b843643a8920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.614094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cda9f-f2d3-4df6-8caa-b843643a8920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.614810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cda9f-f2d3-4df6-8caa-b843643a8920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:02.616403 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cda9f-f2d3-4df6-8caa-b843643a8920 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:02.627149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1300/2587] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:02.631032 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bc304-1c32-4f6d-aa4e-759023a158a2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:02.640636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bc304-1c32-4f6d-aa4e-759023a158a2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:02.645169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10ad0b-30da-4121-8f8b-6d682fb2ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:02.646063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10ad0b-30da-4121-8f8b-6d682fb2ebbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.646346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10ad0b-30da-4121-8f8b-6d682fb2ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.646617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10ad0b-30da-4121-8f8b-6d682fb2ebbd admin admin] PATH_INFO: `/v3/projects/6fd8328ac64349f7846374a8cef3c3a6/users/f5c2b7b891a2475fa514f923d5d3d089/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.660456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10ad0b-30da-4121-8f8b-6d682fb2ebbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd8328ac64349f7846374a8cef3c3a6, user_id=f5c2b7b891a2475fa514f923d5d3d089, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:02.663126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10ad0b-30da-4121-8f8b-6d682fb2ebbd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:02.731777 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1288/2588] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:02 2022] PUT /identity/v3/projects/6fd8328ac64349f7846374a8cef3c3a6/users/f5c2b7b891a2475fa514f923d5d3d089/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:02.743851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba883-c9fa-4f69-be8e-6980057af39d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.744244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba883-c9fa-4f69-be8e-6980057af39d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.744531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba883-c9fa-4f69-be8e-6980057af39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.771351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-715ba883-c9fa-4f69-be8e-6980057af39d None None] MFA Rules not processed for user `f5c2b7b891a2475fa514f923d5d3d089`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:02.772548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-715ba883-c9fa-4f69-be8e-6980057af39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:02.835518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1301/2589] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:02.840037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10ad0b-30da-4121-8f8b-6d682fb2ebbd admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:02.898584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10ad0b-30da-4121-8f8b-6d682fb2ebbd admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:02.903099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd876d-6312-49e4-b7ca-90a567f8e06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:02.903746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd876d-6312-49e4-b7ca-90a567f8e06e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:02.904027 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd876d-6312-49e4-b7ca-90a567f8e06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:02.904297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd876d-6312-49e4-b7ca-90a567f8e06e admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:02.904971 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd876d-6312-49e4-b7ca-90a567f8e06e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:02.907428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd876d-6312-49e4-b7ca-90a567f8e06e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:03.030499 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1289/2590] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:02 2022] POST /identity/v3/projects => generated 405 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:33:03.034793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ba883-c9fa-4f69-be8e-6980057af39d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:03.045102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ba883-c9fa-4f69-be8e-6980057af39d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:03.100227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7fa71-e957-45b3-9a70-49f3fe101881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:03.100889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7fa71-e957-45b3-9a70-49f3fe101881 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:03.101168 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7fa71-e957-45b3-9a70-49f3fe101881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:03.101440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7fa71-e957-45b3-9a70-49f3fe101881 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:03.102160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7fa71-e957-45b3-9a70-49f3fe101881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:03.104064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7fa71-e957-45b3-9a70-49f3fe101881 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:03.257183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1302/2591] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:03 2022] POST /identity/v3/users => generated 309 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:33:03.261054 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd876d-6312-49e4-b7ca-90a567f8e06e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:03.271970 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd876d-6312-49e4-b7ca-90a567f8e06e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:03.276809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7ee17-ea39-4b15-a5d0-ae57378cf354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:03.277510 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ee17-ea39-4b15-a5d0-ae57378cf354 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:03.277883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ee17-ea39-4b15-a5d0-ae57378cf354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:03.278168 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7ee17-ea39-4b15-a5d0-ae57378cf354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:03.279035 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7ee17-ea39-4b15-a5d0-ae57378cf354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:03.280705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7ee17-ea39-4b15-a5d0-ae57378cf354 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:03.291659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1290/2592] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:03.295585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7fa71-e957-45b3-9a70-49f3fe101881 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:03.305507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7fa71-e957-45b3-9a70-49f3fe101881 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:03.310235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77157a7-66ed-4a1f-9716-e7ca50a3f763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:03.311140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77157a7-66ed-4a1f-9716-e7ca50a3f763 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:03.311425 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77157a7-66ed-4a1f-9716-e7ca50a3f763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:03.311699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77157a7-66ed-4a1f-9716-e7ca50a3f763 admin admin] PATH_INFO: `/v3/projects/133d2b096d834db29af5c9d0451e5bad/users/82c88670d15e4d34a58857b97838f8d3/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:03.326470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77157a7-66ed-4a1f-9716-e7ca50a3f763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=133d2b096d834db29af5c9d0451e5bad, user_id=82c88670d15e4d34a58857b97838f8d3, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:03.329648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77157a7-66ed-4a1f-9716-e7ca50a3f763 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:03.406627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1303/2593] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:03 2022] PUT /identity/v3/projects/133d2b096d834db29af5c9d0451e5bad/users/82c88670d15e4d34a58857b97838f8d3/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:33:03.410554 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7ee17-ea39-4b15-a5d0-ae57378cf354 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:03.420733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7ee17-ea39-4b15-a5d0-ae57378cf354 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:03.476413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eb1d8-4370-4ba5-8ea1-ae1292236157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:03.477090 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb1d8-4370-4ba5-8ea1-ae1292236157 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:03.477373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb1d8-4370-4ba5-8ea1-ae1292236157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:03.477651 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb1d8-4370-4ba5-8ea1-ae1292236157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:03.478915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eb1d8-4370-4ba5-8ea1-ae1292236157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:03.480001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eb1d8-4370-4ba5-8ea1-ae1292236157 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:03.489727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1291/2594] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:03.493917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77157a7-66ed-4a1f-9716-e7ca50a3f763 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:03.504721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77157a7-66ed-4a1f-9716-e7ca50a3f763 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:03.509344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:03.510258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:03.510541 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:03.510818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] PATH_INFO: `/v3/projects/133d2b096d834db29af5c9d0451e5bad/users/82c88670d15e4d34a58857b97838f8d3/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:03.512782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=133d2b096d834db29af5c9d0451e5bad, user_id=82c88670d15e4d34a58857b97838f8d3, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:03.516000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:03.525793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1304/2595] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:03 2022] PUT /identity/v3/projects/133d2b096d834db29af5c9d0451e5bad/users/82c88670d15e4d34a58857b97838f8d3/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:03.535760 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f832-307d-4a2c-b1b6-c3211b6d6224 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:03.535992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f832-307d-4a2c-b1b6-c3211b6d6224 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:03.536274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f832-307d-4a2c-b1b6-c3211b6d6224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:03.561449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-7242f832-307d-4a2c-b1b6-c3211b6d6224 None None] MFA Rules not processed for user `82c88670d15e4d34a58857b97838f8d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:03.563221 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7242f832-307d-4a2c-b1b6-c3211b6d6224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:03.628691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1292/2596] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:03.639167 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:03.641365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:03.694649 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be8810-530e-425e-8650-4092dc8c9b01 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:03.698316 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8d1a4-f69e-4283-bacc-dc106f5482dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:03.698960 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8d1a4-f69e-4283-bacc-dc106f5482dc service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:03.699246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8d1a4-f69e-4283-bacc-dc106f5482dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:03.699527 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8d1a4-f69e-4283-bacc-dc106f5482dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:03.709087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8d1a4-f69e-4283-bacc-dc106f5482dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:03.710549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8d1a4-f69e-4283-bacc-dc106f5482dc service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:03.720458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1305/2597] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:03 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:03.861142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f832-307d-4a2c-b1b6-c3211b6d6224 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:03.920154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f832-307d-4a2c-b1b6-c3211b6d6224 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:03.925373 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:03.926271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:03.926564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:03.926844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] PATH_INFO: `/v3/users/f5c2b7b891a2475fa514f923d5d3d089` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:03.928040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c2b7b891a2475fa514f923d5d3d089)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:03.930034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:04.087848 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c2b7b891a2475fa514f923d5d3d089'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.088421 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c2b7b891a2475fa514f923d5d3d089'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.089033 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c2b7b891a2475fa514f923d5d3d089'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.095769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c2b7b891a2475fa514f923d5d3d089'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.142911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c2b7b891a2475fa514f923d5d3d089'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.159966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1293/2598] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:03 2022] DELETE /identity/v3/users/f5c2b7b891a2475fa514f923d5d3d089 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:04.164438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80056a33-ab2e-4908-896a-98d0a20ba03b None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:04.164807 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80056a33-ab2e-4908-896a-98d0a20ba03b None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:04.165208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80056a33-ab2e-4908-896a-98d0a20ba03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:04.183209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-80056a33-ab2e-4908-896a-98d0a20ba03b None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:04.184417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-80056a33-ab2e-4908-896a-98d0a20ba03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:04.243455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1306/2599] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:04.253065 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:04.254487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:04.309148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ca11a-9d44-447b-96e5-e03631d20fb5 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:04.312766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21eb7a2-0887-4777-b30e-c89329e4ea96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:04.313398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eb7a2-0887-4777-b30e-c89329e4ea96 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:04.313681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eb7a2-0887-4777-b30e-c89329e4ea96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:04.313986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eb7a2-0887-4777-b30e-c89329e4ea96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:04.322833 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21eb7a2-0887-4777-b30e-c89329e4ea96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:04.324245 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21eb7a2-0887-4777-b30e-c89329e4ea96 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:04.334002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1294/2600] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:04.699064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80056a33-ab2e-4908-896a-98d0a20ba03b None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:04.700560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-80056a33-ab2e-4908-896a-98d0a20ba03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:04.713635 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80056a33-ab2e-4908-896a-98d0a20ba03b None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:04.719475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:04.720368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:04.720658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:04.720936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] PATH_INFO: `/v3/projects/6fd8328ac64349f7846374a8cef3c3a6` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:04.722963 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd8328ac64349f7846374a8cef3c3a6)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:04.726845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:04.804718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd8328ac64349f7846374a8cef3c3a6', 'request_id': 'req-a907e619-ff20-42d0-b291-cba9845318d8'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.809967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd8328ac64349f7846374a8cef3c3a6', 'request_id': 'req-a907e619-ff20-42d0-b291-cba9845318d8'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.851754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd8328ac64349f7846374a8cef3c3a6', 'request_id': 'req-a907e619-ff20-42d0-b291-cba9845318d8'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.862415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] The token cache is being invalidate because project 6fd8328ac64349f7846374a8cef3c3a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:04.862698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.863315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:04.864838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1307/2601] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:04 2022] DELETE /identity/v3/projects/6fd8328ac64349f7846374a8cef3c3a6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:04.869022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21eb7a2-0887-4777-b30e-c89329e4ea96 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:04.879142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21eb7a2-0887-4777-b30e-c89329e4ea96 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:04.936404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:04.937885 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:04.938184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:04.938463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] PATH_INFO: `/v3/users/82c88670d15e4d34a58857b97838f8d3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:04.939757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c88670d15e4d34a58857b97838f8d3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:04.941694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:05.105962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c88670d15e4d34a58857b97838f8d3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.106691 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c88670d15e4d34a58857b97838f8d3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.107350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c88670d15e4d34a58857b97838f8d3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.114674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c88670d15e4d34a58857b97838f8d3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.160402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-8e57b3ee-fb7d-4138-9bb7-a1dd63ff99cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c88670d15e4d34a58857b97838f8d3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.177619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1295/2602] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:04 2022] DELETE /identity/v3/users/82c88670d15e4d34a58857b97838f8d3 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:05.579679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:05.581117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:05.595026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907e619-ff20-42d0-b291-cba9845318d8 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:05.649016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:05.649838 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:05.650128 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:05.650404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] PATH_INFO: `/v3/projects/133d2b096d834db29af5c9d0451e5bad` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:05.651585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=133d2b096d834db29af5c9d0451e5bad)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:05.653500 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:05.750679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '133d2b096d834db29af5c9d0451e5bad', 'request_id': 'req-1bf87e79-45b0-46cc-8f21-a38436f34b27'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.757061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '133d2b096d834db29af5c9d0451e5bad', 'request_id': 'req-1bf87e79-45b0-46cc-8f21-a38436f34b27'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.840499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '133d2b096d834db29af5c9d0451e5bad', 'request_id': 'req-1bf87e79-45b0-46cc-8f21-a38436f34b27'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.852284 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] The token cache is being invalidate because project 133d2b096d834db29af5c9d0451e5bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:05.852573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.853206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-1bf87e79-45b0-46cc-8f21-a38436f34b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:05.854774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1308/2603] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:05 2022] DELETE /identity/v3/projects/133d2b096d834db29af5c9d0451e5bad => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:05.859753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290f707-dbbe-404f-841b-af23f6108335 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:05.860047 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290f707-dbbe-404f-841b-af23f6108335 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:05.860326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290f707-dbbe-404f-841b-af23f6108335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:05.878682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a290f707-dbbe-404f-841b-af23f6108335 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:05.879954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a290f707-dbbe-404f-841b-af23f6108335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:05.937875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1296/2604] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:05.951004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e53b1-5b68-484b-80a4-bab8bc530060 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:05.951294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e53b1-5b68-484b-80a4-bab8bc530060 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:05.951573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e53b1-5b68-484b-80a4-bab8bc530060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:05.971058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-510e53b1-5b68-484b-80a4-bab8bc530060 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:05.972272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-510e53b1-5b68-484b-80a4-bab8bc530060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:05.980739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1309/2605] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:05.985049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290f707-dbbe-404f-841b-af23f6108335 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:05.994679 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290f707-dbbe-404f-841b-af23f6108335 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:05.999246 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:06.002339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.002339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.002339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.002339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:33:06.002339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:06.003575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:06.014862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1297/2606] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:33:06.020322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e53b1-5b68-484b-80a4-bab8bc530060 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:06.028758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e53b1-5b68-484b-80a4-bab8bc530060 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:06.033532 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466711a-b2af-437c-8a90-496a3b650d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:06.034215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466711a-b2af-437c-8a90-496a3b650d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.034501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466711a-b2af-437c-8a90-496a3b650d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.034778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466711a-b2af-437c-8a90-496a3b650d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.035458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466711a-b2af-437c-8a90-496a3b650d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:06.037354 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466711a-b2af-437c-8a90-496a3b650d03 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:06.146410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1310/2607] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:06 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:06.150664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:06.160703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2320218-1e3c-4312-be75-d5c06f708e5a admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:06.215169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1d703-ca14-4c46-9e94-354f24f4b137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:06.215761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1d703-ca14-4c46-9e94-354f24f4b137 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.216049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1d703-ca14-4c46-9e94-354f24f4b137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.216331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1d703-ca14-4c46-9e94-354f24f4b137 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.217026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1d703-ca14-4c46-9e94-354f24f4b137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:06.219017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1d703-ca14-4c46-9e94-354f24f4b137 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:06.355160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1298/2608] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:06 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:06.359761 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466711a-b2af-437c-8a90-496a3b650d03 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:06.370715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466711a-b2af-437c-8a90-496a3b650d03 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:06.375979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5bffa-558e-4b4c-a09d-b6bcdbbe401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:06.383460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5bffa-558e-4b4c-a09d-b6bcdbbe401c admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.383460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5bffa-558e-4b4c-a09d-b6bcdbbe401c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.383460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5bffa-558e-4b4c-a09d-b6bcdbbe401c admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.383460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5bffa-558e-4b4c-a09d-b6bcdbbe401c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:06.383460 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5bffa-558e-4b4c-a09d-b6bcdbbe401c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:06.389506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1311/2609] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:06.393481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1d703-ca14-4c46-9e94-354f24f4b137 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:06.406875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1d703-ca14-4c46-9e94-354f24f4b137 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:06.408609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3190593-31a4-4581-b899-7e363a566863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:06.409269 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3190593-31a4-4581-b899-7e363a566863 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.409559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3190593-31a4-4581-b899-7e363a566863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.409894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3190593-31a4-4581-b899-7e363a566863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.410671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3190593-31a4-4581-b899-7e363a566863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:06.412320 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3190593-31a4-4581-b899-7e363a566863 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:06.424429 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1299/2610] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:06.428743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5bffa-558e-4b4c-a09d-b6bcdbbe401c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:06.439450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5bffa-558e-4b4c-a09d-b6bcdbbe401c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:06.444011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9ce0-1b83-4ba7-8ed3-960ddba87ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:06.444885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9ce0-1b83-4ba7-8ed3-960ddba87ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.445171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9ce0-1b83-4ba7-8ed3-960ddba87ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.445450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a9ce0-1b83-4ba7-8ed3-960ddba87ae4 admin admin] PATH_INFO: `/v3/projects/1becea289c09498888e13ca2981b416b/users/a9c4c340d0ca454a94a5ad65836547d4/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.461573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a9ce0-1b83-4ba7-8ed3-960ddba87ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1becea289c09498888e13ca2981b416b, user_id=a9c4c340d0ca454a94a5ad65836547d4, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:06.464324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a9ce0-1b83-4ba7-8ed3-960ddba87ae4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:06.543567 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1312/2611] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:06 2022] PUT /identity/v3/projects/1becea289c09498888e13ca2981b416b/users/a9c4c340d0ca454a94a5ad65836547d4/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:33:06.555376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493a81-7bfb-4d71-a4bd-578e32d3924e None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.555717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493a81-7bfb-4d71-a4bd-578e32d3924e None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.556007 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493a81-7bfb-4d71-a4bd-578e32d3924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.583726 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ae493a81-7bfb-4d71-a4bd-578e32d3924e None None] MFA Rules not processed for user `a9c4c340d0ca454a94a5ad65836547d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:06.584948 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ae493a81-7bfb-4d71-a4bd-578e32d3924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:06.648124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1300/2612] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:06.652533 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9ce0-1b83-4ba7-8ed3-960ddba87ae4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:06.714511 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a9ce0-1b83-4ba7-8ed3-960ddba87ae4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:06.719793 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e9f4-9b90-41ad-b3e6-aa1b5eeb6160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:06.720449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e9f4-9b90-41ad-b3e6-aa1b5eeb6160 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.720745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e9f4-9b90-41ad-b3e6-aa1b5eeb6160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.721008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e9f4-9b90-41ad-b3e6-aa1b5eeb6160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.721691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262e9f4-9b90-41ad-b3e6-aa1b5eeb6160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:06.723645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262e9f4-9b90-41ad-b3e6-aa1b5eeb6160 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:06.807675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1313/2613] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:06 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:06.811664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae493a81-7bfb-4d71-a4bd-578e32d3924e None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:06.822671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae493a81-7bfb-4d71-a4bd-578e32d3924e None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:06.877277 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5553f-e12c-4227-9154-16e1a9f9e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:06.877978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5553f-e12c-4227-9154-16e1a9f9e62c admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:06.878260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5553f-e12c-4227-9154-16e1a9f9e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:06.878536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5553f-e12c-4227-9154-16e1a9f9e62c admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:06.879223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5553f-e12c-4227-9154-16e1a9f9e62c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:06.881187 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5553f-e12c-4227-9154-16e1a9f9e62c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:07.000142 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1301/2614] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:06 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:07.004148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e9f4-9b90-41ad-b3e6-aa1b5eeb6160 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:07.014658 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e9f4-9b90-41ad-b3e6-aa1b5eeb6160 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:07.019986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb4887-a3e1-441f-8d95-65390678c4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:07.020642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb4887-a3e1-441f-8d95-65390678c4ec admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:07.020927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb4887-a3e1-441f-8d95-65390678c4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:07.021209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb4887-a3e1-441f-8d95-65390678c4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:07.021989 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb4887-a3e1-441f-8d95-65390678c4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:07.023598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb4887-a3e1-441f-8d95-65390678c4ec admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:07.039144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1314/2615] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:07.045175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5553f-e12c-4227-9154-16e1a9f9e62c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:07.050262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5553f-e12c-4227-9154-16e1a9f9e62c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:07.054929 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53115603-d8f3-48a1-b23c-ca959b5a2761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:07.055797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53115603-d8f3-48a1-b23c-ca959b5a2761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:07.056079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53115603-d8f3-48a1-b23c-ca959b5a2761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:07.056354 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53115603-d8f3-48a1-b23c-ca959b5a2761 admin admin] PATH_INFO: `/v3/projects/f6fcce6caa9e467989e48d43fcc88f36/users/e435413ffb8946cf9ec70dbbf229e018/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:07.070642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53115603-d8f3-48a1-b23c-ca959b5a2761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fcce6caa9e467989e48d43fcc88f36, user_id=e435413ffb8946cf9ec70dbbf229e018, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:07.073317 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53115603-d8f3-48a1-b23c-ca959b5a2761 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:07.195445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1302/2616] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:07 2022] PUT /identity/v3/projects/f6fcce6caa9e467989e48d43fcc88f36/users/e435413ffb8946cf9ec70dbbf229e018/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:33:07.202399 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb4887-a3e1-441f-8d95-65390678c4ec admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:07.212959 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb4887-a3e1-441f-8d95-65390678c4ec admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:07.267859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410faef7-0271-4ee3-b16e-f6b8ae281416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:07.268526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410faef7-0271-4ee3-b16e-f6b8ae281416 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:07.268811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410faef7-0271-4ee3-b16e-f6b8ae281416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:07.269089 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410faef7-0271-4ee3-b16e-f6b8ae281416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:07.269837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410faef7-0271-4ee3-b16e-f6b8ae281416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:07.271445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410faef7-0271-4ee3-b16e-f6b8ae281416 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:07.280901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1315/2617] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:07.284985 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53115603-d8f3-48a1-b23c-ca959b5a2761 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:07.295279 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53115603-d8f3-48a1-b23c-ca959b5a2761 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:07.310998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:07.311889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:07.312842 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:07.313124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] PATH_INFO: `/v3/projects/f6fcce6caa9e467989e48d43fcc88f36/users/e435413ffb8946cf9ec70dbbf229e018/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:07.318020 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fcce6caa9e467989e48d43fcc88f36, user_id=e435413ffb8946cf9ec70dbbf229e018, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:07.320974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:07.332196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1303/2618] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:07 2022] PUT /identity/v3/projects/f6fcce6caa9e467989e48d43fcc88f36/users/e435413ffb8946cf9ec70dbbf229e018/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:07.342964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e96ddf-49fd-4bf4-b1c3-ca7c0de53f93 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:07.343315 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e96ddf-49fd-4bf4-b1c3-ca7c0de53f93 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:07.343604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e96ddf-49fd-4bf4-b1c3-ca7c0de53f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:07.369267 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-c2e96ddf-49fd-4bf4-b1c3-ca7c0de53f93 None None] MFA Rules not processed for user `e435413ffb8946cf9ec70dbbf229e018`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:07.370605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c2e96ddf-49fd-4bf4-b1c3-ca7c0de53f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:07.442087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1316/2619] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:07.454475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:07.456302 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:07.511134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698726-9044-46b2-b249-087ff572d754 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:07.514696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f195f34-ce94-4bbf-8aa7-fbdbd7f75845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:07.515989 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f195f34-ce94-4bbf-8aa7-fbdbd7f75845 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:07.516275 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f195f34-ce94-4bbf-8aa7-fbdbd7f75845 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:07.516569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f195f34-ce94-4bbf-8aa7-fbdbd7f75845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:07.525179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f195f34-ce94-4bbf-8aa7-fbdbd7f75845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:07.526626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f195f34-ce94-4bbf-8aa7-fbdbd7f75845 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:07.535825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1304/2620] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:07.564220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e96ddf-49fd-4bf4-b1c3-ca7c0de53f93 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:07.623601 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e96ddf-49fd-4bf4-b1c3-ca7c0de53f93 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:07.628673 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:07.629544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:07.629955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:07.630234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] PATH_INFO: `/v3/users/a9c4c340d0ca454a94a5ad65836547d4` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:07.631438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c4c340d0ca454a94a5ad65836547d4)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:07.633410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:07.910933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c4c340d0ca454a94a5ad65836547d4'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:07.911581 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c4c340d0ca454a94a5ad65836547d4'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.006422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c4c340d0ca454a94a5ad65836547d4'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.013727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c4c340d0ca454a94a5ad65836547d4'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.014393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c4c340d0ca454a94a5ad65836547d4'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.031611 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1317/2621] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:07 2022] DELETE /identity/v3/users/a9c4c340d0ca454a94a5ad65836547d4 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:08.039818 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ee71-049f-4467-a290-b6deed8f10b2 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:08.040303 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ee71-049f-4467-a290-b6deed8f10b2 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:08.040398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2ee71-049f-4467-a290-b6deed8f10b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:08.059960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-b4d2ee71-049f-4467-a290-b6deed8f10b2 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:08.062121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b4d2ee71-049f-4467-a290-b6deed8f10b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:08.126044 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1305/2622] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:08.136628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:08.138292 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:08.192884 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbc763-43f5-4175-a3b8-527dddc66b98 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:08.196345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee34b6-d89a-4575-b9a6-a89bbf4faa73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:08.196974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee34b6-d89a-4575-b9a6-a89bbf4faa73 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:08.197254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee34b6-d89a-4575-b9a6-a89bbf4faa73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:08.197525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee34b6-d89a-4575-b9a6-a89bbf4faa73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:08.206227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee34b6-d89a-4575-b9a6-a89bbf4faa73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:08.207642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee34b6-d89a-4575-b9a6-a89bbf4faa73 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:08.217210 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1318/2623] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:08.619398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2ee71-049f-4467-a290-b6deed8f10b2 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:08.627850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b4d2ee71-049f-4467-a290-b6deed8f10b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:08.632672 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2ee71-049f-4467-a290-b6deed8f10b2 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:08.637248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:08.638127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:08.638408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:08.638682 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] PATH_INFO: `/v3/projects/1becea289c09498888e13ca2981b416b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:08.639883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1becea289c09498888e13ca2981b416b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:08.642360 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:08.762389 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1becea289c09498888e13ca2981b416b', 'request_id': 'req-0e99c021-0105-4e8a-a7dd-4a89426474e3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.771408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1becea289c09498888e13ca2981b416b', 'request_id': 'req-0e99c021-0105-4e8a-a7dd-4a89426474e3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.853559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1becea289c09498888e13ca2981b416b', 'request_id': 'req-0e99c021-0105-4e8a-a7dd-4a89426474e3'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.874332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] The token cache is being invalidate because project 1becea289c09498888e13ca2981b416b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:08.875113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.875807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:08.877412 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1306/2624] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:08 2022] DELETE /identity/v3/projects/1becea289c09498888e13ca2981b416b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:08.881363 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee34b6-d89a-4575-b9a6-a89bbf4faa73 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:08.892478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee34b6-d89a-4575-b9a6-a89bbf4faa73 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:08.946865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:08.947768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:08.948057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:08.948345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] PATH_INFO: `/v3/users/e435413ffb8946cf9ec70dbbf229e018` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:08.949548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e435413ffb8946cf9ec70dbbf229e018)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:08.951539 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:09.135213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e435413ffb8946cf9ec70dbbf229e018'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.135778 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e435413ffb8946cf9ec70dbbf229e018'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.223230 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e435413ffb8946cf9ec70dbbf229e018'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.230272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e435413ffb8946cf9ec70dbbf229e018'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.230947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-884a5559-8d91-4cfe-a076-e572a74b853b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e435413ffb8946cf9ec70dbbf229e018'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.248271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1319/2625] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:08 2022] DELETE /identity/v3/users/e435413ffb8946cf9ec70dbbf229e018 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:09.642147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:09.643959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:09.659241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99c021-0105-4e8a-a7dd-4a89426474e3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:09.716711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:09.716904 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:09.717191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:09.717468 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] PATH_INFO: `/v3/projects/f6fcce6caa9e467989e48d43fcc88f36` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:09.719194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fcce6caa9e467989e48d43fcc88f36)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:09.721111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:09.818395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fcce6caa9e467989e48d43fcc88f36', 'request_id': 'req-f804dab5-c85c-4ebe-9af3-1746f139ce01'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.823857 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fcce6caa9e467989e48d43fcc88f36', 'request_id': 'req-f804dab5-c85c-4ebe-9af3-1746f139ce01'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.892054 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fcce6caa9e467989e48d43fcc88f36', 'request_id': 'req-f804dab5-c85c-4ebe-9af3-1746f139ce01'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.904204 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] The token cache is being invalidate because project f6fcce6caa9e467989e48d43fcc88f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:09.904487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.905252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-f804dab5-c85c-4ebe-9af3-1746f139ce01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:09.906804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1307/2626] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:09 2022] DELETE /identity/v3/projects/f6fcce6caa9e467989e48d43fcc88f36 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:33:09.912864 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b121c-25ec-401d-acb5-c6a574e6d190 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:09.913104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b121c-25ec-401d-acb5-c6a574e6d190 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:09.913418 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b121c-25ec-401d-acb5-c6a574e6d190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:09.931379 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-bc2b121c-25ec-401d-acb5-c6a574e6d190 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:09.932587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-bc2b121c-25ec-401d-acb5-c6a574e6d190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:09.990535 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1320/2627] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:09.998621 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece4514-4d7f-4d0f-9a0e-f5ef3cabf985 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:09.998914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece4514-4d7f-4d0f-9a0e-f5ef3cabf985 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:09.999226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece4514-4d7f-4d0f-9a0e-f5ef3cabf985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.018407 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bece4514-4d7f-4d0f-9a0e-f5ef3cabf985 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:10.019655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bece4514-4d7f-4d0f-9a0e-f5ef3cabf985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:10.027521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1308/2628] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:10.031856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b121c-25ec-401d-acb5-c6a574e6d190 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:10.041469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b121c-25ec-401d-acb5-c6a574e6d190 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:10.046086 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:10.047227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:10.047512 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:10.047786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.048503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:33:10.048811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:10.050453 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:10.061454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1321/2629] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:33:10.066415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece4514-4d7f-4d0f-9a0e-f5ef3cabf985 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:10.075352 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece4514-4d7f-4d0f-9a0e-f5ef3cabf985 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:10.079955 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc0935-9748-4def-89ab-d702ab77da0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:10.080593 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc0935-9748-4def-89ab-d702ab77da0e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:10.080873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc0935-9748-4def-89ab-d702ab77da0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:10.081150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc0935-9748-4def-89ab-d702ab77da0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.081870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc0935-9748-4def-89ab-d702ab77da0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:10.084023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc0935-9748-4def-89ab-d702ab77da0e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:10.196898 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1309/2630] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:10 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:10.200977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:10.211086 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542b577-524c-4b89-a716-8e477362905e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:10.268003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7c8fd-4869-483c-842d-044e2b7da356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:10.268676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7c8fd-4869-483c-842d-044e2b7da356 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:10.268962 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7c8fd-4869-483c-842d-044e2b7da356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:10.269240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7c8fd-4869-483c-842d-044e2b7da356 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.270101 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7c8fd-4869-483c-842d-044e2b7da356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:10.272030 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7c8fd-4869-483c-842d-044e2b7da356 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:10.354771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1322/2631] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:10 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:10.359640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc0935-9748-4def-89ab-d702ab77da0e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:10.370178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc0935-9748-4def-89ab-d702ab77da0e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:10.374926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f303db-2f60-423b-bb42-eaa31d740f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:10.375584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f303db-2f60-423b-bb42-eaa31d740f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:10.375873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f303db-2f60-423b-bb42-eaa31d740f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:10.376155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f303db-2f60-423b-bb42-eaa31d740f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.376963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f303db-2f60-423b-bb42-eaa31d740f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:10.378603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f303db-2f60-423b-bb42-eaa31d740f6f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:10.388787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1310/2632] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:10.398380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7c8fd-4869-483c-842d-044e2b7da356 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:10.409019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7c8fd-4869-483c-842d-044e2b7da356 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:10.413986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6cd02-7b98-462a-a593-5b57edae194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:10.414653 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6cd02-7b98-462a-a593-5b57edae194a admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:10.414938 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6cd02-7b98-462a-a593-5b57edae194a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:10.415215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6cd02-7b98-462a-a593-5b57edae194a admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.416107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6cd02-7b98-462a-a593-5b57edae194a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:10.417703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6cd02-7b98-462a-a593-5b57edae194a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:10.428283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1323/2633] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:10.432445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f303db-2f60-423b-bb42-eaa31d740f6f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:10.442192 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f303db-2f60-423b-bb42-eaa31d740f6f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:10.446809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:10.447719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:10.447944 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:10.448217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] PATH_INFO: `/v3/projects/7aa81ae3f0d340fd8d30752efbf7f557/users/edd7e25efa014993b430f57b867db765/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.463124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa81ae3f0d340fd8d30752efbf7f557, user_id=edd7e25efa014993b430f57b867db765, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:10.465754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:10.543903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1311/2634] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:10 2022] PUT /identity/v3/projects/7aa81ae3f0d340fd8d30752efbf7f557/users/edd7e25efa014993b430f57b867db765/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:10.555754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeb520-3e76-4ec9-8cbd-865484e33f5d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:10.556053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeb520-3e76-4ec9-8cbd-865484e33f5d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:10.556334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeb520-3e76-4ec9-8cbd-865484e33f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.582233 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-7dbeb520-3e76-4ec9-8cbd-865484e33f5d None None] MFA Rules not processed for user `edd7e25efa014993b430f57b867db765`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:10.583681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7dbeb520-3e76-4ec9-8cbd-865484e33f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:10.644347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1324/2635] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:10.656379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:10.658119 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:10.713533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4962c5-3736-46e2-a218-0a413c7c9649 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:10.717105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c11ec4-dca9-49eb-810d-472f974fc51b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:10.718223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11ec4-dca9-49eb-810d-472f974fc51b service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:10.718223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11ec4-dca9-49eb-810d-472f974fc51b service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:10.719064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11ec4-dca9-49eb-810d-472f974fc51b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:10.728176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c11ec4-dca9-49eb-810d-472f974fc51b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:10.729715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c11ec4-dca9-49eb-810d-472f974fc51b service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:10.739493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1312/2636] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:14.896895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeb520-3e76-4ec9-8cbd-865484e33f5d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:14.956665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeb520-3e76-4ec9-8cbd-865484e33f5d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:14.961501 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:14.962417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:14.962698 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:14.962974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] PATH_INFO: `/v3/users/edd7e25efa014993b430f57b867db765` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:14.964171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd7e25efa014993b430f57b867db765)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:14.966143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:15.150749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd7e25efa014993b430f57b867db765'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:15.151337 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd7e25efa014993b430f57b867db765'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:15.223503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd7e25efa014993b430f57b867db765'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:15.230381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd7e25efa014993b430f57b867db765'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:15.231009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd7e25efa014993b430f57b867db765'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:15.247454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1325/2637] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:14 2022] DELETE /identity/v3/users/edd7e25efa014993b430f57b867db765 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:15.251926 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46597d-c3c7-4255-8db2-7051b24c4fda None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:15.252396 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46597d-c3c7-4255-8db2-7051b24c4fda None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:15.252526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46597d-c3c7-4255-8db2-7051b24c4fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:15.269457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-9c46597d-c3c7-4255-8db2-7051b24c4fda None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:15.270663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9c46597d-c3c7-4255-8db2-7051b24c4fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:15.325582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1313/2638] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:15.335841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:15.337302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:15.396822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57048cf6-9fc0-4551-947e-15e760bdb826 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:15.400503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73500252-f3c4-45f2-b9f6-0dc21580b5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:15.401203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73500252-f3c4-45f2-b9f6-0dc21580b5de service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:15.401435 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73500252-f3c4-45f2-b9f6-0dc21580b5de service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:15.401693 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73500252-f3c4-45f2-b9f6-0dc21580b5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:15.411108 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73500252-f3c4-45f2-b9f6-0dc21580b5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:15.412538 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73500252-f3c4-45f2-b9f6-0dc21580b5de service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:15.421808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1326/2639] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:15.878394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46597d-c3c7-4255-8db2-7051b24c4fda None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:15.879130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-9c46597d-c3c7-4255-8db2-7051b24c4fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:15.893255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46597d-c3c7-4255-8db2-7051b24c4fda None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:15.899772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:15.900883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:15.901173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:15.901452 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] PATH_INFO: `/v3/projects/7aa81ae3f0d340fd8d30752efbf7f557` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:15.903331 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa81ae3f0d340fd8d30752efbf7f557)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:15.905274 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:16.006260 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa81ae3f0d340fd8d30752efbf7f557', 'request_id': 'req-3a2377de-445a-4083-a673-090c8c792359'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:16.011414 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa81ae3f0d340fd8d30752efbf7f557', 'request_id': 'req-3a2377de-445a-4083-a673-090c8c792359'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:16.109283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa81ae3f0d340fd8d30752efbf7f557', 'request_id': 'req-3a2377de-445a-4083-a673-090c8c792359'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:16.121681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] The token cache is being invalidate because project 7aa81ae3f0d340fd8d30752efbf7f557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:16.121999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:16.122637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:16.124179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1314/2640] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:15 2022] DELETE /identity/v3/projects/7aa81ae3f0d340fd8d30752efbf7f557 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:16.129680 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996453f-0dae-48f1-a8be-62ba8ef027fb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.130186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996453f-0dae-48f1-a8be-62ba8ef027fb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.130340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996453f-0dae-48f1-a8be-62ba8ef027fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.149431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2996453f-0dae-48f1-a8be-62ba8ef027fb None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:16.150682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2996453f-0dae-48f1-a8be-62ba8ef027fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:16.159615 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:16.161083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:16.211492 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1327/2641] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:16.218782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e31ff5-0933-4fb0-89fe-33a0c0ad50fb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.219076 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e31ff5-0933-4fb0-89fe-33a0c0ad50fb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.219356 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e31ff5-0933-4fb0-89fe-33a0c0ad50fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.232438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2377de-445a-4083-a673-090c8c792359 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:16.238401 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-26e31ff5-0933-4fb0-89fe-33a0c0ad50fb None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:16.238825 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '310c5d3ba81044ac8f3f192728cc3e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:16.239526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.239661 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-26e31ff5-0933-4fb0-89fe-33a0c0ad50fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:16.239815 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.240102 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.242092 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:16.247796 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1328/2642] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:16.251646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e31ff5-0933-4fb0-89fe-33a0c0ad50fb None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:16.254271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:16.256167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:16.260409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e31ff5-0933-4fb0-89fe-33a0c0ad50fb None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:16.264941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:16.265576 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.265898 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.266178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.266727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1315/2643] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:33:16.266982 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:33:16.267240 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:16.268966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:16.280571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1329/2644] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:33:16.285315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:16.295537 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00736db8-2863-420a-a406-49db5ba3a74e service placement] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:16.301002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd80f7-531e-4a5d-9eb4-d01e30ad8704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:16.301662 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd80f7-531e-4a5d-9eb4-d01e30ad8704 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.302070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd80f7-531e-4a5d-9eb4-d01e30ad8704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.302350 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd80f7-531e-4a5d-9eb4-d01e30ad8704 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.303058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd80f7-531e-4a5d-9eb4-d01e30ad8704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:16.305001 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd80f7-531e-4a5d-9eb4-d01e30ad8704 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:16.481116 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1316/2645] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:16 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:33:16.481968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:16.492396 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6ade9-2fb6-42ce-9a23-dc421984faa4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:16.549345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49336d-6bbb-47db-91a7-d33e5a48d978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:16.550059 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49336d-6bbb-47db-91a7-d33e5a48d978 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.550343 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49336d-6bbb-47db-91a7-d33e5a48d978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.550615 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49336d-6bbb-47db-91a7-d33e5a48d978 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.551308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49336d-6bbb-47db-91a7-d33e5a48d978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:16.553222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49336d-6bbb-47db-91a7-d33e5a48d978 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:16.634411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1330/2646] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:16 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:16.638470 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd80f7-531e-4a5d-9eb4-d01e30ad8704 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:16.649104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd80f7-531e-4a5d-9eb4-d01e30ad8704 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:16.653733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d6445-66b8-4a70-a5c8-e6233af351ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:16.654433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d6445-66b8-4a70-a5c8-e6233af351ec admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.654722 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d6445-66b8-4a70-a5c8-e6233af351ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.655002 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d6445-66b8-4a70-a5c8-e6233af351ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.655726 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d6445-66b8-4a70-a5c8-e6233af351ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:16.657319 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d6445-66b8-4a70-a5c8-e6233af351ec admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:16.668294 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1317/2647] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:16.672083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49336d-6bbb-47db-91a7-d33e5a48d978 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:16.681652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49336d-6bbb-47db-91a7-d33e5a48d978 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:16.686550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aba3b5-27b5-4344-97ac-daeca31b4b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:16.687200 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aba3b5-27b5-4344-97ac-daeca31b4b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.687512 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aba3b5-27b5-4344-97ac-daeca31b4b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.687786 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aba3b5-27b5-4344-97ac-daeca31b4b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.688526 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aba3b5-27b5-4344-97ac-daeca31b4b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:16.690146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aba3b5-27b5-4344-97ac-daeca31b4b14 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:16.699626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1331/2648] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:16.703518 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d6445-66b8-4a70-a5c8-e6233af351ec admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:16.713911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d6445-66b8-4a70-a5c8-e6233af351ec admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:16.718476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835866ba-133f-4353-8b98-c64fb09c5a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:16.719339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835866ba-133f-4353-8b98-c64fb09c5a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.719619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835866ba-133f-4353-8b98-c64fb09c5a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.719949 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835866ba-133f-4353-8b98-c64fb09c5a85 admin admin] PATH_INFO: `/v3/projects/a931f5dd55a04f85ab8ed35b9a86a4a3/users/3eaf289c57a4461d91ce74bab74a87d2/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.734337 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835866ba-133f-4353-8b98-c64fb09c5a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a931f5dd55a04f85ab8ed35b9a86a4a3, user_id=3eaf289c57a4461d91ce74bab74a87d2, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:16.736986 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835866ba-133f-4353-8b98-c64fb09c5a85 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:16.795547 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1318/2649] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:16 2022] PUT /identity/v3/projects/a931f5dd55a04f85ab8ed35b9a86a4a3/users/3eaf289c57a4461d91ce74bab74a87d2/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:16.805801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd813d58-8fc7-4876-a92a-4ea2fff5a8b0 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.805801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd813d58-8fc7-4876-a92a-4ea2fff5a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.806064 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd813d58-8fc7-4876-a92a-4ea2fff5a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.833405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-cd813d58-8fc7-4876-a92a-4ea2fff5a8b0 None None] MFA Rules not processed for user `3eaf289c57a4461d91ce74bab74a87d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:16.834634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-cd813d58-8fc7-4876-a92a-4ea2fff5a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:16.896473 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1332/2650] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:16.900916 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835866ba-133f-4353-8b98-c64fb09c5a85 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:16.962422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835866ba-133f-4353-8b98-c64fb09c5a85 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:16.967159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042ff0b-afe2-4634-a551-468492c27d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:16.967808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042ff0b-afe2-4634-a551-468492c27d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:16.968086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042ff0b-afe2-4634-a551-468492c27d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:16.968361 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042ff0b-afe2-4634-a551-468492c27d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:16.969048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042ff0b-afe2-4634-a551-468492c27d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:16.970983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042ff0b-afe2-4634-a551-468492c27d57 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:17.076381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1319/2651] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:16 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:17.080823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd813d58-8fc7-4876-a92a-4ea2fff5a8b0 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:17.091675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd813d58-8fc7-4876-a92a-4ea2fff5a8b0 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:17.145431 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ece65-3c0d-4792-8a86-dda6af6f98e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:17.146173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ece65-3c0d-4792-8a86-dda6af6f98e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.146414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ece65-3c0d-4792-8a86-dda6af6f98e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.146695 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ece65-3c0d-4792-8a86-dda6af6f98e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:17.147382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ece65-3c0d-4792-8a86-dda6af6f98e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:17.149310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ece65-3c0d-4792-8a86-dda6af6f98e5 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:17.235087 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1333/2652] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:17 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:17.239557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042ff0b-afe2-4634-a551-468492c27d57 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:17.249748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042ff0b-afe2-4634-a551-468492c27d57 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:17.254601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf141a5-8078-406a-bced-bc542afa57ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:17.255256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf141a5-8078-406a-bced-bc542afa57ee admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.255693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf141a5-8078-406a-bced-bc542afa57ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.256007 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf141a5-8078-406a-bced-bc542afa57ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:17.256757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf141a5-8078-406a-bced-bc542afa57ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:17.258382 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf141a5-8078-406a-bced-bc542afa57ee admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:17.268637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1320/2653] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:17.272603 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ece65-3c0d-4792-8a86-dda6af6f98e5 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:17.283029 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ece65-3c0d-4792-8a86-dda6af6f98e5 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:17.287659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6d18e-3c55-45c4-8412-ae678e332c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:17.288309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6d18e-3c55-45c4-8412-ae678e332c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.288595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6d18e-3c55-45c4-8412-ae678e332c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.288875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6d18e-3c55-45c4-8412-ae678e332c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:17.289623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6d18e-3c55-45c4-8412-ae678e332c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:17.291275 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6d18e-3c55-45c4-8412-ae678e332c7a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:17.301368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1334/2654] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:17.305479 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf141a5-8078-406a-bced-bc542afa57ee admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:17.315664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf141a5-8078-406a-bced-bc542afa57ee admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:17.320340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd224408-d14c-4aa5-81eb-03feb70aef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:17.321209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd224408-d14c-4aa5-81eb-03feb70aef6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.321490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd224408-d14c-4aa5-81eb-03feb70aef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.321764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd224408-d14c-4aa5-81eb-03feb70aef6e admin admin] PATH_INFO: `/v3/projects/0c4ad78234ba424e8c04f53fc265af41/users/080590607f81402e83855e7aa037b9b1/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:17.337229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd224408-d14c-4aa5-81eb-03feb70aef6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4ad78234ba424e8c04f53fc265af41, user_id=080590607f81402e83855e7aa037b9b1, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:17.340006 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd224408-d14c-4aa5-81eb-03feb70aef6e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:17.451852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1321/2655] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:17 2022] PUT /identity/v3/projects/0c4ad78234ba424e8c04f53fc265af41/users/080590607f81402e83855e7aa037b9b1/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:17.463876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed03618-180e-4018-b605-a42c1a875bc7 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.464179 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed03618-180e-4018-b605-a42c1a875bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.464462 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed03618-180e-4018-b605-a42c1a875bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:17.492554 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6ed03618-180e-4018-b605-a42c1a875bc7 None None] MFA Rules not processed for user `080590607f81402e83855e7aa037b9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:17.497312 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6ed03618-180e-4018-b605-a42c1a875bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:17.555310 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1335/2656] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:17.559994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd224408-d14c-4aa5-81eb-03feb70aef6e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:17.621385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd224408-d14c-4aa5-81eb-03feb70aef6e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:17.625984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b35b1a-398a-473b-bb6a-22a672d7f46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:17.626637 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b35b1a-398a-473b-bb6a-22a672d7f46f admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.626917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b35b1a-398a-473b-bb6a-22a672d7f46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.627228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b35b1a-398a-473b-bb6a-22a672d7f46f admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:17.627889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b35b1a-398a-473b-bb6a-22a672d7f46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:17.629882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b35b1a-398a-473b-bb6a-22a672d7f46f admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:17.773655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1322/2657] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:17 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:17.777578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed03618-180e-4018-b605-a42c1a875bc7 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:17.789326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed03618-180e-4018-b605-a42c1a875bc7 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:17.843502 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cfe86-7d14-4566-8922-ca933e4c017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:17.844174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cfe86-7d14-4566-8922-ca933e4c017e admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.844457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cfe86-7d14-4566-8922-ca933e4c017e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.844732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cfe86-7d14-4566-8922-ca933e4c017e admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:17.845454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cfe86-7d14-4566-8922-ca933e4c017e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:17.847409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cfe86-7d14-4566-8922-ca933e4c017e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:17.932368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1336/2658] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:17 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:17.936416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b35b1a-398a-473b-bb6a-22a672d7f46f admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:17.947223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b35b1a-398a-473b-bb6a-22a672d7f46f admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:17.951816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6205d4-2aa9-4c30-aac7-514fce50ec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:17.952475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6205d4-2aa9-4c30-aac7-514fce50ec3b admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.952761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6205d4-2aa9-4c30-aac7-514fce50ec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.953039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6205d4-2aa9-4c30-aac7-514fce50ec3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:17.953765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6205d4-2aa9-4c30-aac7-514fce50ec3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:17.955966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6205d4-2aa9-4c30-aac7-514fce50ec3b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:17.966498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1323/2659] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:17.970581 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cfe86-7d14-4566-8922-ca933e4c017e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:17.980241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cfe86-7d14-4566-8922-ca933e4c017e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:17.984768 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:17.985637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:17.985956 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:17.986234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] PATH_INFO: `/v3/projects/795ca8a323a547fbb43d6b4797bf276f/users/e75a935790694e1e8157d3f2f4209a2f/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:18.000084 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=795ca8a323a547fbb43d6b4797bf276f, user_id=e75a935790694e1e8157d3f2f4209a2f, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:18.003277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:18.062709 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1337/2660] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:17 2022] PUT /identity/v3/projects/795ca8a323a547fbb43d6b4797bf276f/users/e75a935790694e1e8157d3f2f4209a2f/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:18.067224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6205d4-2aa9-4c30-aac7-514fce50ec3b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:18.077147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6205d4-2aa9-4c30-aac7-514fce50ec3b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:18.094039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:18.095760 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:18.136400 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d28ee0-0f04-4030-8474-6444eb068fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:18.139498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d28ee0-0f04-4030-8474-6444eb068fea admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:18.139498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d28ee0-0f04-4030-8474-6444eb068fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:18.139498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d28ee0-0f04-4030-8474-6444eb068fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:18.139498 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d28ee0-0f04-4030-8474-6444eb068fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:18.140146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d28ee0-0f04-4030-8474-6444eb068fea admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:18.149914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1324/2661] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:18.153974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d28ee0-0f04-4030-8474-6444eb068fea admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:18.161160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d15d00-dbd1-4da8-aecf-bf12fd46af83 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:18.163750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d28ee0-0f04-4030-8474-6444eb068fea admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:18.166021 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:18.166897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:18.167180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:18.167464 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] PATH_INFO: `/v3/users/3ea3e6b8869448d9a9bc40f985b90de1` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:18.168668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea3e6b8869448d9a9bc40f985b90de1)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:18.168809 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5fed7-f893-4ed4-aaa9-b9b1bfe7234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:18.169629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5fed7-f893-4ed4-aaa9-b9b1bfe7234c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:18.169956 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5fed7-f893-4ed4-aaa9-b9b1bfe7234c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:18.170344 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5fed7-f893-4ed4-aaa9-b9b1bfe7234c admin admin] PATH_INFO: `/v3/projects/795ca8a323a547fbb43d6b4797bf276f/users/e75a935790694e1e8157d3f2f4209a2f/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:18.170622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:18.172464 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5fed7-f893-4ed4-aaa9-b9b1bfe7234c admin admin] RBAC: Authorizing `identity:create_grant(project_id=795ca8a323a547fbb43d6b4797bf276f, user_id=e75a935790694e1e8157d3f2f4209a2f, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:18.175922 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5fed7-f893-4ed4-aaa9-b9b1bfe7234c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:18.186258 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1325/2662] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:18 2022] PUT /identity/v3/projects/795ca8a323a547fbb43d6b4797bf276f/users/e75a935790694e1e8157d3f2f4209a2f/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:33:18.196832 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283827a3-157f-4e93-bac0-85b5be04af36 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:18.197127 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283827a3-157f-4e93-bac0-85b5be04af36 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:18.197410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283827a3-157f-4e93-bac0-85b5be04af36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:18.223803 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-283827a3-157f-4e93-bac0-85b5be04af36 None None] MFA Rules not processed for user `e75a935790694e1e8157d3f2f4209a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:18.225010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-283827a3-157f-4e93-bac0-85b5be04af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:18.290213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1326/2663] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:33:18.302059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283827a3-157f-4e93-bac0-85b5be04af36 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:18.303447 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-283827a3-157f-4e93-bac0-85b5be04af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:18.361441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283827a3-157f-4e93-bac0-85b5be04af36 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:18.365089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d898c-4840-47e2-83e1-c44656084502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:18.365739 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d898c-4840-47e2-83e1-c44656084502 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:18.366071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d898c-4840-47e2-83e1-c44656084502 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:18.366340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d898c-4840-47e2-83e1-c44656084502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:18.380523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea3e6b8869448d9a9bc40f985b90de1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:18.381199 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea3e6b8869448d9a9bc40f985b90de1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:18.422849 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d898c-4840-47e2-83e1-c44656084502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:18.424924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d898c-4840-47e2-83e1-c44656084502 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:18.433831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1327/2664] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:33:18.469208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea3e6b8869448d9a9bc40f985b90de1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:18.477465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea3e6b8869448d9a9bc40f985b90de1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:18.478270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea3e6b8869448d9a9bc40f985b90de1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:18.496939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1338/2665] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:18 2022] DELETE /identity/v3/users/3ea3e6b8869448d9a9bc40f985b90de1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:33:18.501176 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4af87-96ff-4033-ac8b-cd5b59638612 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:18.501636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4af87-96ff-4033-ac8b-cd5b59638612 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:18.501961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4af87-96ff-4033-ac8b-cd5b59638612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:18.518967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-ade4af87-96ff-4033-ac8b-cd5b59638612 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:18.520232 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ade4af87-96ff-4033-ac8b-cd5b59638612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:18.579428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1328/2666] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:18.589404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:18.590817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:18.645623 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2c535-b9af-46c0-acb2-b7dc87357f2a admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:18.649241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e0da0-ddd8-46ed-8216-97d260d189e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:18.649928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0da0-ddd8-46ed-8216-97d260d189e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:18.650217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0da0-ddd8-46ed-8216-97d260d189e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:18.650518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0da0-ddd8-46ed-8216-97d260d189e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:18.659814 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0da0-ddd8-46ed-8216-97d260d189e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:18.661246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0da0-ddd8-46ed-8216-97d260d189e1 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:18.670550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1339/2667] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:19.054270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4af87-96ff-4033-ac8b-cd5b59638612 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:19.055712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ade4af87-96ff-4033-ac8b-cd5b59638612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:19.080820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4af87-96ff-4033-ac8b-cd5b59638612 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:19.085963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:19.089113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.089113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.089113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] PATH_INFO: `/v3/projects/0ebe87e3f2c54a34aa824ce542d24cfa` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.089113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebe87e3f2c54a34aa824ce542d24cfa)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:19.090860 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:19.218533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebe87e3f2c54a34aa824ce542d24cfa', 'request_id': 'req-95fd2cb4-b0c9-4686-9800-f526846f66f1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:19.223787 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebe87e3f2c54a34aa824ce542d24cfa', 'request_id': 'req-95fd2cb4-b0c9-4686-9800-f526846f66f1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:19.315191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebe87e3f2c54a34aa824ce542d24cfa', 'request_id': 'req-95fd2cb4-b0c9-4686-9800-f526846f66f1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:19.326408 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] The token cache is being invalidate because project 0ebe87e3f2c54a34aa824ce542d24cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:19.326693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:19.327338 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-95fd2cb4-b0c9-4686-9800-f526846f66f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:19.328875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1329/2668] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:19 2022] DELETE /identity/v3/projects/0ebe87e3f2c54a34aa824ce542d24cfa => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:19.335003 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175718-1d77-4c6e-8b42-93ca5a005d3e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.335506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175718-1d77-4c6e-8b42-93ca5a005d3e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.335613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175718-1d77-4c6e-8b42-93ca5a005d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.353893 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-79175718-1d77-4c6e-8b42-93ca5a005d3e None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:19.355219 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-79175718-1d77-4c6e-8b42-93ca5a005d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:19.414198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1340/2669] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:19.422552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee334dc-3368-4e28-b7c2-9387840a209f None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.422845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee334dc-3368-4e28-b7c2-9387840a209f None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.423129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee334dc-3368-4e28-b7c2-9387840a209f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.442070 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-5ee334dc-3368-4e28-b7c2-9387840a209f None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:19.443292 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5ee334dc-3368-4e28-b7c2-9387840a209f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:19.451384 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1330/2670] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:19.456575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79175718-1d77-4c6e-8b42-93ca5a005d3e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:19.466797 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79175718-1d77-4c6e-8b42-93ca5a005d3e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:19.471486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:19.472140 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.472878 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.473159 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.473920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:33:19.474231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:19.475844 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:19.487304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1341/2671] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:33:19.491502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee334dc-3368-4e28-b7c2-9387840a209f None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:19.500879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee334dc-3368-4e28-b7c2-9387840a209f None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:19.506111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864f97b-b61a-48aa-a3ef-7df083bfab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:19.506766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f97b-b61a-48aa-a3ef-7df083bfab3e admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.507051 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f97b-b61a-48aa-a3ef-7df083bfab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.507327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f97b-b61a-48aa-a3ef-7df083bfab3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.508030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864f97b-b61a-48aa-a3ef-7df083bfab3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:19.509970 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864f97b-b61a-48aa-a3ef-7df083bfab3e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:19.579285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1331/2672] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:19 2022] POST /identity/v3/projects => generated 365 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:33:19.583467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:19.593583 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b41f-6d15-4e57-a70e-e6c7152b0132 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:19.648366 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248c557-0d89-4f3f-bfdb-db4202e3fcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:19.649034 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c557-0d89-4f3f-bfdb-db4202e3fcdc admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.649322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c557-0d89-4f3f-bfdb-db4202e3fcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.649642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c557-0d89-4f3f-bfdb-db4202e3fcdc admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.650457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248c557-0d89-4f3f-bfdb-db4202e3fcdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:19.652381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248c557-0d89-4f3f-bfdb-db4202e3fcdc admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:19.749190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1342/2673] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:19 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:19.753206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864f97b-b61a-48aa-a3ef-7df083bfab3e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:19.763369 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864f97b-b61a-48aa-a3ef-7df083bfab3e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:19.768533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4c710-9bf9-42f8-b776-d41974af7e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:19.769189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c710-9bf9-42f8-b776-d41974af7e66 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.769476 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c710-9bf9-42f8-b776-d41974af7e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.769757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4c710-9bf9-42f8-b776-d41974af7e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.770548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4c710-9bf9-42f8-b776-d41974af7e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:19.772189 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4c710-9bf9-42f8-b776-d41974af7e66 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:19.782330 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1332/2674] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:19.790053 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248c557-0d89-4f3f-bfdb-db4202e3fcdc admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:19.800051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248c557-0d89-4f3f-bfdb-db4202e3fcdc admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:19.804660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de797e-5d75-4b15-be73-8a389a8d4b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:19.805403 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de797e-5d75-4b15-be73-8a389a8d4b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.805707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de797e-5d75-4b15-be73-8a389a8d4b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.806005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de797e-5d75-4b15-be73-8a389a8d4b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.806727 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de797e-5d75-4b15-be73-8a389a8d4b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:19.808331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de797e-5d75-4b15-be73-8a389a8d4b2a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:19.818165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1343/2675] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:19.824612 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4c710-9bf9-42f8-b776-d41974af7e66 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:19.835247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4c710-9bf9-42f8-b776-d41974af7e66 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:19.839846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:19.840714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.841000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.841281 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] PATH_INFO: `/v3/projects/76dd54851aa84efc8a947cdfe5e0c99f/users/655098e9487b41c9876d41e1f0d07791/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.856077 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76dd54851aa84efc8a947cdfe5e0c99f, user_id=655098e9487b41c9876d41e1f0d07791, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:19.858797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:19.926678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1333/2676] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:19 2022] PUT /identity/v3/projects/76dd54851aa84efc8a947cdfe5e0c99f/users/655098e9487b41c9876d41e1f0d07791/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:19.936941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1891ad-24b2-4813-b5d3-e387b59fc7ec None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:19.937239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1891ad-24b2-4813-b5d3-e387b59fc7ec None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:19.937523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1891ad-24b2-4813-b5d3-e387b59fc7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:19.966079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-1a1891ad-24b2-4813-b5d3-e387b59fc7ec None None] MFA Rules not processed for user `655098e9487b41c9876d41e1f0d07791`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:19.967301 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1a1891ad-24b2-4813-b5d3-e387b59fc7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:20.028876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1344/2677] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:20.040379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:20.042104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:20.097066 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f56760-6aa8-465b-a115-0f7ba9d31d2d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:20.100627 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd52f7-33d7-4b46-a8dd-141e3a20e68c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:20.101257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd52f7-33d7-4b46-a8dd-141e3a20e68c service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:20.101537 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd52f7-33d7-4b46-a8dd-141e3a20e68c service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:20.101831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd52f7-33d7-4b46-a8dd-141e3a20e68c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:20.109874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd52f7-33d7-4b46-a8dd-141e3a20e68c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:20.111327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd52f7-33d7-4b46-a8dd-141e3a20e68c service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:20.119841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1334/2678] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:20.469454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1891ad-24b2-4813-b5d3-e387b59fc7ec None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:20.470947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-1a1891ad-24b2-4813-b5d3-e387b59fc7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:20.482969 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1891ad-24b2-4813-b5d3-e387b59fc7ec None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:20.487025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:20.487665 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:20.487946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:20.488225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:20.489980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:20.547531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:20.549474 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:20.559106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1345/2679] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:22.379069 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd52f7-33d7-4b46-a8dd-141e3a20e68c service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:22.387433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd52f7-33d7-4b46-a8dd-141e3a20e68c service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:22.391061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fea115-b869-4bd7-bfe8-4e28f9524c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:22.391689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fea115-b869-4bd7-bfe8-4e28f9524c61 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:22.392043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fea115-b869-4bd7-bfe8-4e28f9524c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:22.392315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fea115-b869-4bd7-bfe8-4e28f9524c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:22.394087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-82fea115-b869-4bd7-bfe8-4e28f9524c61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:22.456191 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fea115-b869-4bd7-bfe8-4e28f9524c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:22.457625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fea115-b869-4bd7-bfe8-4e28f9524c61 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:22.467229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1335/2680] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:27.619074 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:27.620513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:27.683642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a68987-85aa-4d13-b038-4a30469071ea service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:27.688173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:27.689031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:27.689859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:27.690148 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] PATH_INFO: `/v3/users/3eaf289c57a4461d91ce74bab74a87d2` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:27.691393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eaf289c57a4461d91ce74bab74a87d2)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:27.693322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:27.904504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eaf289c57a4461d91ce74bab74a87d2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:27.905081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eaf289c57a4461d91ce74bab74a87d2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:27.970998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eaf289c57a4461d91ce74bab74a87d2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:27.978537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eaf289c57a4461d91ce74bab74a87d2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:27.979190 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eaf289c57a4461d91ce74bab74a87d2'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:27.996082 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1346/2681] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:27 2022] DELETE /identity/v3/users/3eaf289c57a4461d91ce74bab74a87d2 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:28.000619 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694675-5a68-454a-ad83-e08eefa90704 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:28.001112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694675-5a68-454a-ad83-e08eefa90704 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:28.001208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694675-5a68-454a-ad83-e08eefa90704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:28.029938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-bd694675-5a68-454a-ad83-e08eefa90704 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:28.031173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd694675-5a68-454a-ad83-e08eefa90704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:28.092686 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1336/2682] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:28.102445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:28.103900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:28.160459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75e73-ca8b-4e9b-9f58-805aff777e04 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:28.164067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a06f6c-a9f0-41f3-8d2d-355a76158e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:28.164716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06f6c-a9f0-41f3-8d2d-355a76158e83 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:28.165055 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06f6c-a9f0-41f3-8d2d-355a76158e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:28.165340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06f6c-a9f0-41f3-8d2d-355a76158e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:28.175587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a06f6c-a9f0-41f3-8d2d-355a76158e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:28.177121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a06f6c-a9f0-41f3-8d2d-355a76158e83 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:28.186668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1347/2683] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:28.602572 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd694675-5a68-454a-ad83-e08eefa90704 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:28.608828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-bd694675-5a68-454a-ad83-e08eefa90704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:28.615572 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd694675-5a68-454a-ad83-e08eefa90704 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:28.620160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:28.621005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:28.621284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:28.621558 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] PATH_INFO: `/v3/projects/a931f5dd55a04f85ab8ed35b9a86a4a3` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:28.622758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a931f5dd55a04f85ab8ed35b9a86a4a3)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:28.624680 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:28.749911 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a931f5dd55a04f85ab8ed35b9a86a4a3', 'request_id': 'req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:28.761434 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a931f5dd55a04f85ab8ed35b9a86a4a3', 'request_id': 'req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:28.876355 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a931f5dd55a04f85ab8ed35b9a86a4a3', 'request_id': 'req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:28.886043 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] The token cache is being invalidate because project a931f5dd55a04f85ab8ed35b9a86a4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:28.886333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:28.887008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:28.888568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1337/2684] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:28 2022] DELETE /identity/v3/projects/a931f5dd55a04f85ab8ed35b9a86a4a3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:28.893093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a06f6c-a9f0-41f3-8d2d-355a76158e83 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:28.902926 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a06f6c-a9f0-41f3-8d2d-355a76158e83 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:28.961608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:28.962537 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:28.962825 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:28.963106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] PATH_INFO: `/v3/users/080590607f81402e83855e7aa037b9b1` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:28.964411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=080590607f81402e83855e7aa037b9b1)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:28.966389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:29.221171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080590607f81402e83855e7aa037b9b1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:29.221888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080590607f81402e83855e7aa037b9b1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:29.309384 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080590607f81402e83855e7aa037b9b1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:29.317172 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080590607f81402e83855e7aa037b9b1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:29.317839 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080590607f81402e83855e7aa037b9b1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:29.337755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1348/2685] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:28 2022] DELETE /identity/v3/users/080590607f81402e83855e7aa037b9b1 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:29.862137 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:29.863624 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:29.876257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c17b6-c49b-43f4-ba26-e53f5ff6ed21 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:29.935812 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:29.936685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:29.936973 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:29.937252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] PATH_INFO: `/v3/projects/0c4ad78234ba424e8c04f53fc265af41` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:29.938472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4ad78234ba424e8c04f53fc265af41)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:29.940401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:30.071727 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4ad78234ba424e8c04f53fc265af41', 'request_id': 'req-121549e7-818f-47f0-873d-09d5bc48f4d2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.077040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4ad78234ba424e8c04f53fc265af41', 'request_id': 'req-121549e7-818f-47f0-873d-09d5bc48f4d2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.148962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4ad78234ba424e8c04f53fc265af41', 'request_id': 'req-121549e7-818f-47f0-873d-09d5bc48f4d2'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.160871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] The token cache is being invalidate because project 0c4ad78234ba424e8c04f53fc265af41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:30.161150 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.161770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.163356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1338/2686] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:29 2022] DELETE /identity/v3/projects/0c4ad78234ba424e8c04f53fc265af41 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:30.167444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:30.168865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:30.181524 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca1117-3a7b-4106-a78f-af46c0353ac9 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:30.236339 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:30.237207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:30.237544 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:30.237851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] PATH_INFO: `/v3/users/e75a935790694e1e8157d3f2f4209a2f` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:30.239095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75a935790694e1e8157d3f2f4209a2f)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:30.241060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:30.442978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75a935790694e1e8157d3f2f4209a2f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.443645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75a935790694e1e8157d3f2f4209a2f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.531961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75a935790694e1e8157d3f2f4209a2f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.539438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75a935790694e1e8157d3f2f4209a2f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.540071 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-41c70d03-b760-4b0d-9a80-3d707e7de0e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75a935790694e1e8157d3f2f4209a2f'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:30.557113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1349/2687] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:30 2022] DELETE /identity/v3/users/e75a935790694e1e8157d3f2f4209a2f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:31.009462 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:31.011600 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:31.025721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121549e7-818f-47f0-873d-09d5bc48f4d2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:31.080871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:31.081730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.082045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.082326 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] PATH_INFO: `/v3/projects/795ca8a323a547fbb43d6b4797bf276f` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.083761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=795ca8a323a547fbb43d6b4797bf276f)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:31.085770 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:31.285866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795ca8a323a547fbb43d6b4797bf276f', 'request_id': 'req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:31.291093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795ca8a323a547fbb43d6b4797bf276f', 'request_id': 'req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:31.416405 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795ca8a323a547fbb43d6b4797bf276f', 'request_id': 'req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:31.428215 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] The token cache is being invalidate because project 795ca8a323a547fbb43d6b4797bf276f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:31.435431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:31.435431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2360bd5c-ed51-4004-be8c-ffcaaf0759c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:31.435431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1339/2688] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:31 2022] DELETE /identity/v3/projects/795ca8a323a547fbb43d6b4797bf276f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:31.435790 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e3690-9fca-42e6-bd85-3ae750c7665c None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.436227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e3690-9fca-42e6-bd85-3ae750c7665c None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.436227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e3690-9fca-42e6-bd85-3ae750c7665c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.453189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-fe6e3690-9fca-42e6-bd85-3ae750c7665c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:31.454403 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-fe6e3690-9fca-42e6-bd85-3ae750c7665c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:31.514144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1350/2689] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:31.523222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb041ec-c2ff-404a-b2bf-bb1a2a168619 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.523514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb041ec-c2ff-404a-b2bf-bb1a2a168619 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.523801 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb041ec-c2ff-404a-b2bf-bb1a2a168619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.542449 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8fb041ec-c2ff-404a-b2bf-bb1a2a168619 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:31.543570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8fb041ec-c2ff-404a-b2bf-bb1a2a168619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:31.551531 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1340/2690] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:31.555626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e3690-9fca-42e6-bd85-3ae750c7665c None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:31.565291 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e3690-9fca-42e6-bd85-3ae750c7665c None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:31.569857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:31.570525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.570806 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.571079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.571794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:33:31.572098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:31.573699 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:31.585357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1351/2691] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:33:31.589358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb041ec-c2ff-404a-b2bf-bb1a2a168619 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:31.599173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb041ec-c2ff-404a-b2bf-bb1a2a168619 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:31.604014 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b38f6-8467-4e85-9c4c-e63e03ab1483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:31.604663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b38f6-8467-4e85-9c4c-e63e03ab1483 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.604950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b38f6-8467-4e85-9c4c-e63e03ab1483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.605229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b38f6-8467-4e85-9c4c-e63e03ab1483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.605959 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b38f6-8467-4e85-9c4c-e63e03ab1483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:31.607866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b38f6-8467-4e85-9c4c-e63e03ab1483 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:31.726768 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1341/2692] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:31 2022] POST /identity/v3/projects => generated 391 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:31.730833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:31.740676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a01908-3adb-4742-a0ca-37cf945f5eba admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:31.795480 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747708f3-e560-4fad-8428-f11c9258fe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:31.796075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747708f3-e560-4fad-8428-f11c9258fe71 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.796364 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747708f3-e560-4fad-8428-f11c9258fe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.796646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747708f3-e560-4fad-8428-f11c9258fe71 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.797365 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747708f3-e560-4fad-8428-f11c9258fe71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:31.799348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747708f3-e560-4fad-8428-f11c9258fe71 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:31.886211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1352/2693] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:31 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:33:31.890446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b38f6-8467-4e85-9c4c-e63e03ab1483 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:31.900906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b38f6-8467-4e85-9c4c-e63e03ab1483 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:31.905534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245f79-7e82-4bd3-ab42-d7da019601c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:31.906266 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245f79-7e82-4bd3-ab42-d7da019601c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.906503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245f79-7e82-4bd3-ab42-d7da019601c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.906776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245f79-7e82-4bd3-ab42-d7da019601c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.907515 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50245f79-7e82-4bd3-ab42-d7da019601c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:31.909668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50245f79-7e82-4bd3-ab42-d7da019601c3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:31.920117 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1342/2694] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:31.924117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747708f3-e560-4fad-8428-f11c9258fe71 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:31.934070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747708f3-e560-4fad-8428-f11c9258fe71 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:31.938749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b4035-24b9-4963-9322-e3d25b8c3cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:31.939940 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4035-24b9-4963-9322-e3d25b8c3cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.940227 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4035-24b9-4963-9322-e3d25b8c3cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.940627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4035-24b9-4963-9322-e3d25b8c3cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.941360 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b4035-24b9-4963-9322-e3d25b8c3cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:31.942996 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b4035-24b9-4963-9322-e3d25b8c3cc1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:31.953417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1353/2695] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:31.958226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245f79-7e82-4bd3-ab42-d7da019601c3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:31.968228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245f79-7e82-4bd3-ab42-d7da019601c3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:31.972778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:31.973648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:31.973964 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:31.974234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] PATH_INFO: `/v3/projects/9149f4cc999c4422917026ccafc7f3b9/users/d33f04a48a3b498ea1d375bf219f7e27/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:31.987968 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9149f4cc999c4422917026ccafc7f3b9, user_id=d33f04a48a3b498ea1d375bf219f7e27, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:31.990648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:32.063840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1343/2696] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:31 2022] PUT /identity/v3/projects/9149f4cc999c4422917026ccafc7f3b9/users/d33f04a48a3b498ea1d375bf219f7e27/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:33:32.074660 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f018e-42ab-4c0a-8245-27453a95d968 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:32.074955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f018e-42ab-4c0a-8245-27453a95d968 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:32.075237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f018e-42ab-4c0a-8245-27453a95d968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:32.105335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-7b1f018e-42ab-4c0a-8245-27453a95d968 None None] MFA Rules not processed for user `d33f04a48a3b498ea1d375bf219f7e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:32.106608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7b1f018e-42ab-4c0a-8245-27453a95d968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:32.167835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1354/2697] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:32.178422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:32.180089 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:32.233813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e96037-e0a6-4537-9aa0-6f648fa7ef00 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:32.237392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b7de6-ec84-4f78-8b45-8b4b109bbe66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:32.238058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7de6-ec84-4f78-8b45-8b4b109bbe66 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:32.238336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7de6-ec84-4f78-8b45-8b4b109bbe66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:32.238617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b7de6-ec84-4f78-8b45-8b4b109bbe66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:32.246484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b7de6-ec84-4f78-8b45-8b4b109bbe66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:32.247902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b7de6-ec84-4f78-8b45-8b4b109bbe66 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:32.256858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1344/2698] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:48.843688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418068ff-b272-461e-bbb0-7437d37296aa None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:48.844189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418068ff-b272-461e-bbb0-7437d37296aa None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:48.844277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418068ff-b272-461e-bbb0-7437d37296aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:48.863385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-418068ff-b272-461e-bbb0-7437d37296aa None None] MFA Rules not processed for user `79dbde2a65224927af4bb5e77090496c`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:48.864575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-418068ff-b272-461e-bbb0-7437d37296aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:48.920177 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1355/2699] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:55.282747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b7de6-ec84-4f78-8b45-8b4b109bbe66 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:55.348296 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b7de6-ec84-4f78-8b45-8b4b109bbe66 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:55.353046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:55.354022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:55.354378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:55.354724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] PATH_INFO: `/v3/users/d33f04a48a3b498ea1d375bf219f7e27` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:55.355924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33f04a48a3b498ea1d375bf219f7e27)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:55.357923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:55.534309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33f04a48a3b498ea1d375bf219f7e27'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:55.535454 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33f04a48a3b498ea1d375bf219f7e27'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:55.536194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33f04a48a3b498ea1d375bf219f7e27'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:55.542958 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33f04a48a3b498ea1d375bf219f7e27'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:55.594957 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33f04a48a3b498ea1d375bf219f7e27'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:55.615461 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1345/2700] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:33:55 2022] DELETE /identity/v3/users/d33f04a48a3b498ea1d375bf219f7e27 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:55.620090 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a64369-8457-40f6-9965-c7cffa8b6f6a None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:55.620717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a64369-8457-40f6-9965-c7cffa8b6f6a None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:55.620717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a64369-8457-40f6-9965-c7cffa8b6f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:55.638427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-16a64369-8457-40f6-9965-c7cffa8b6f6a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:55.639647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-16a64369-8457-40f6-9965-c7cffa8b6f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:55.697368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1356/2701] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:55.709115 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:55.710577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:55.767716 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dffa5-2113-4241-86cc-b0aca11b3eac admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:55.771938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512e721-4947-4c9b-a549-6640a6154a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:55.772588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512e721-4947-4c9b-a549-6640a6154a53 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:55.772874 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512e721-4947-4c9b-a549-6640a6154a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:55.773155 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512e721-4947-4c9b-a549-6640a6154a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:55.782098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512e721-4947-4c9b-a549-6640a6154a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:55.783606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512e721-4947-4c9b-a549-6640a6154a53 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:55.794433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1346/2702] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:33:56.209263 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a64369-8457-40f6-9965-c7cffa8b6f6a None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:56.210856 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-16a64369-8457-40f6-9965-c7cffa8b6f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:56.223421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a64369-8457-40f6-9965-c7cffa8b6f6a None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:56.228350 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:56.242069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:56.242069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:56.242069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] PATH_INFO: `/v3/projects/9149f4cc999c4422917026ccafc7f3b9` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:56.242069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9149f4cc999c4422917026ccafc7f3b9)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:56.242069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:56.344479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9149f4cc999c4422917026ccafc7f3b9', 'request_id': 'req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:56.349667 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9149f4cc999c4422917026ccafc7f3b9', 'request_id': 'req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:56.433728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9149f4cc999c4422917026ccafc7f3b9', 'request_id': 'req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:56.447517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] The token cache is being invalidate because project 9149f4cc999c4422917026ccafc7f3b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:33:56.448111 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:56.448599 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ea66ab2e-04e1-4bfa-91f0-cf214f925b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:33:56.450294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1357/2703] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:33:56 2022] DELETE /identity/v3/projects/9149f4cc999c4422917026ccafc7f3b9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:56.455242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ea1f8-56f9-4820-a7b0-a8b11667fc1a None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:56.455741 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ea1f8-56f9-4820-a7b0-a8b11667fc1a None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:56.455850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ea1f8-56f9-4820-a7b0-a8b11667fc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:56.473141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-141ea1f8-56f9-4820-a7b0-a8b11667fc1a None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:56.474385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-141ea1f8-56f9-4820-a7b0-a8b11667fc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:56.535622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1347/2704] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:56.543889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9130a5-e5ed-4587-b715-9dcd4b50f2fb None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:56.544180 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9130a5-e5ed-4587-b715-9dcd4b50f2fb None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:56.544461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9130a5-e5ed-4587-b715-9dcd4b50f2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:56.563993 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-8d9130a5-e5ed-4587-b715-9dcd4b50f2fb None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:56.565204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8d9130a5-e5ed-4587-b715-9dcd4b50f2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:56.573759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1358/2705] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:56.577966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ea1f8-56f9-4820-a7b0-a8b11667fc1a None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:56.588050 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ea1f8-56f9-4820-a7b0-a8b11667fc1a None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:56.592718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:56.593381 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:56.593808 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:56.594098 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:56.595042 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:33:56.595290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:56.596879 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:56.613493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1348/2706] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:33:56.614707 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9130a5-e5ed-4587-b715-9dcd4b50f2fb None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:56.624812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9130a5-e5ed-4587-b715-9dcd4b50f2fb None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:56.629570 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b865a2-cf7c-4f49-9b2b-9aac6e4c6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:56.630272 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b865a2-cf7c-4f49-9b2b-9aac6e4c6c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:56.630563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b865a2-cf7c-4f49-9b2b-9aac6e4c6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:56.630840 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b865a2-cf7c-4f49-9b2b-9aac6e4c6c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:56.631617 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b865a2-cf7c-4f49-9b2b-9aac6e4c6c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:56.633445 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b865a2-cf7c-4f49-9b2b-9aac6e4c6c4c admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:56.762774 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1359/2707] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:33:56 2022] POST /identity/v3/projects => generated 397 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:56.767000 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:56.777427 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bb2d5-1c81-4a86-934c-beb142737516 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:56.839708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534228e-4e40-467b-9f98-e4106524a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:56.840388 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534228e-4e40-467b-9f98-e4106524a2da admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:56.840681 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534228e-4e40-467b-9f98-e4106524a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:56.841009 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534228e-4e40-467b-9f98-e4106524a2da admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:56.841816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534228e-4e40-467b-9f98-e4106524a2da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:56.843817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534228e-4e40-467b-9f98-e4106524a2da admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:56.967208 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1349/2708] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:33:56 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:33:56.971266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b865a2-cf7c-4f49-9b2b-9aac6e4c6c4c admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:56.981845 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b865a2-cf7c-4f49-9b2b-9aac6e4c6c4c admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:56.986335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3b036-c6fb-4516-a750-b9a877de05c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:56.986992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3b036-c6fb-4516-a750-b9a877de05c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:56.987404 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3b036-c6fb-4516-a750-b9a877de05c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:56.987562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3b036-c6fb-4516-a750-b9a877de05c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:56.988785 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3b036-c6fb-4516-a750-b9a877de05c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:56.990447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3b036-c6fb-4516-a750-b9a877de05c5 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:57.004392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1360/2709] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:57.005936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534228e-4e40-467b-9f98-e4106524a2da admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:57.016018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534228e-4e40-467b-9f98-e4106524a2da admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:57.020754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b15042-accc-41be-933e-c49863dadb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:57.021482 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15042-accc-41be-933e-c49863dadb3a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:57.021767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15042-accc-41be-933e-c49863dadb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:57.022126 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15042-accc-41be-933e-c49863dadb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:57.022924 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b15042-accc-41be-933e-c49863dadb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:57.024532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b15042-accc-41be-933e-c49863dadb3a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:57.034502 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1350/2710] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:33:57.038801 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3b036-c6fb-4516-a750-b9a877de05c5 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:57.052070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3b036-c6fb-4516-a750-b9a877de05c5 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:57.056733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:57.057622 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:57.057941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:57.058239 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] PATH_INFO: `/v3/projects/cd76ca7c77c14fd4938eccc53c0d7fb3/users/0bba904f4ffe4c8fbe45835ae2628d09/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:57.073716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd76ca7c77c14fd4938eccc53c0d7fb3, user_id=0bba904f4ffe4c8fbe45835ae2628d09, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:57.076427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:57.161600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1361/2711] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:33:57 2022] PUT /identity/v3/projects/cd76ca7c77c14fd4938eccc53c0d7fb3/users/0bba904f4ffe4c8fbe45835ae2628d09/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:33:57.171867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f9de-9a5f-4086-86b1-441bdf5736f7 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:57.172166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f9de-9a5f-4086-86b1-441bdf5736f7 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:57.172445 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f9de-9a5f-4086-86b1-441bdf5736f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:57.200900 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8a25f9de-9a5f-4086-86b1-441bdf5736f7 None None] MFA Rules not processed for user `0bba904f4ffe4c8fbe45835ae2628d09`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:33:57.206458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8a25f9de-9a5f-4086-86b1-441bdf5736f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:57.265733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1351/2712] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:33:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:33:57.278250 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:33:57.279946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:33:57.337427 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b16f09-3296-4dae-b803-9b2679d04e21 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:33:57.349579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e436e-f980-44fe-8736-4b5cc8d6d0fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:33:57.352348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e436e-f980-44fe-8736-4b5cc8d6d0fd service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:33:57.352348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e436e-f980-44fe-8736-4b5cc8d6d0fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:33:57.352348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e436e-f980-44fe-8736-4b5cc8d6d0fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:33:57.359924 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e436e-f980-44fe-8736-4b5cc8d6d0fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:33:57.361389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e436e-f980-44fe-8736-4b5cc8d6d0fd service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:33:57.371195 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1362/2713] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:33:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:34:06.354674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25f9de-9a5f-4086-86b1-441bdf5736f7 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:06.421852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25f9de-9a5f-4086-86b1-441bdf5736f7 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:06.427049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:06.428053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:06.428335 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:06.428606 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] PATH_INFO: `/v3/users/0bba904f4ffe4c8fbe45835ae2628d09` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:06.429892 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bba904f4ffe4c8fbe45835ae2628d09)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:06.431827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:06.661004 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bba904f4ffe4c8fbe45835ae2628d09'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:06.661683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bba904f4ffe4c8fbe45835ae2628d09'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:06.662392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bba904f4ffe4c8fbe45835ae2628d09'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:06.670710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bba904f4ffe4c8fbe45835ae2628d09'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:06.732295 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bba904f4ffe4c8fbe45835ae2628d09'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:06.751578 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1352/2714] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:34:06 2022] DELETE /identity/v3/users/0bba904f4ffe4c8fbe45835ae2628d09 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:34:06.756334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932cec1-140d-4560-a7e2-b333e60fd0a8 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:06.756806 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932cec1-140d-4560-a7e2-b333e60fd0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:06.757015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932cec1-140d-4560-a7e2-b333e60fd0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:06.775585 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-9932cec1-140d-4560-a7e2-b333e60fd0a8 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:34:06.777248 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9932cec1-140d-4560-a7e2-b333e60fd0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:06.856578 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1363/2715] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:34:06.866135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:06.867603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:06.923903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbec635-f2dc-4c67-97af-edcacdfdc4b0 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:06.928596 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:06.930478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:06.930779 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:06.931111 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:06.943022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:06.945551 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:06.956549 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1353/2716] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:34:07.391433 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932cec1-140d-4560-a7e2-b333e60fd0a8 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:07.393162 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-9932cec1-140d-4560-a7e2-b333e60fd0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:07.407489 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932cec1-140d-4560-a7e2-b333e60fd0a8 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:07.413482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:07.415069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:07.415069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:07.415069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] PATH_INFO: `/v3/projects/cd76ca7c77c14fd4938eccc53c0d7fb3` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:07.416346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd76ca7c77c14fd4938eccc53c0d7fb3)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:07.418458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:07.541106 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd76ca7c77c14fd4938eccc53c0d7fb3', 'request_id': 'req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:07.546514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd76ca7c77c14fd4938eccc53c0d7fb3', 'request_id': 'req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:07.650542 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd76ca7c77c14fd4938eccc53c0d7fb3', 'request_id': 'req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:07.662435 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] The token cache is being invalidate because project cd76ca7c77c14fd4938eccc53c0d7fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:34:07.662723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:07.663352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-beeb8a25-04f5-4544-8bf4-5df5a9e9225e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:07.664885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1364/2717] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:34:07 2022] DELETE /identity/v3/projects/cd76ca7c77c14fd4938eccc53c0d7fb3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:34:10.781463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:10.782950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:10.797827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c11b9d-913b-41f6-9e8b-0911eb6d6006 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:10.859772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:10.860645 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:10.860929 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:10.861210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] PATH_INFO: `/v3/users/655098e9487b41c9876d41e1f0d07791` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:10.862422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=655098e9487b41c9876d41e1f0d07791)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:10.864356 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:11.076131 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655098e9487b41c9876d41e1f0d07791'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.076772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655098e9487b41c9876d41e1f0d07791'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.077438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655098e9487b41c9876d41e1f0d07791'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.085230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655098e9487b41c9876d41e1f0d07791'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.160230 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655098e9487b41c9876d41e1f0d07791'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.178410 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1354/2718] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:34:10 2022] DELETE /identity/v3/users/655098e9487b41c9876d41e1f0d07791 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:34:11.182896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23302-4a4a-47da-9fc7-0636725047a1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:11.183371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23302-4a4a-47da-9fc7-0636725047a1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:11.183487 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23302-4a4a-47da-9fc7-0636725047a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:11.204701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-68a23302-4a4a-47da-9fc7-0636725047a1 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:34:11.207870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-68a23302-4a4a-47da-9fc7-0636725047a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:11.264329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1365/2719] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:34:11.273940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:11.275395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:11.330852 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100c2b0-2f3d-4e0f-9635-f9b4d62209ee admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:11.334781 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55c6dc-ca74-4912-a7fc-240ddad31661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:11.335428 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55c6dc-ca74-4912-a7fc-240ddad31661 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:11.335754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55c6dc-ca74-4912-a7fc-240ddad31661 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:11.336034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55c6dc-ca74-4912-a7fc-240ddad31661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:11.344945 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55c6dc-ca74-4912-a7fc-240ddad31661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:11.346424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55c6dc-ca74-4912-a7fc-240ddad31661 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:11.355655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1355/2720] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:34:11.727043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23302-4a4a-47da-9fc7-0636725047a1 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:11.728579 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-68a23302-4a4a-47da-9fc7-0636725047a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:11.742015 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23302-4a4a-47da-9fc7-0636725047a1 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:11.746629 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:11.748114 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:11.748457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:11.748663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] PATH_INFO: `/v3/projects/76dd54851aa84efc8a947cdfe5e0c99f` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:11.750038 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76dd54851aa84efc8a947cdfe5e0c99f)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:11.751937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:11.885648 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76dd54851aa84efc8a947cdfe5e0c99f', 'request_id': 'req-256ececb-0625-46a3-ad5f-3069c2da4048'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.890965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76dd54851aa84efc8a947cdfe5e0c99f', 'request_id': 'req-256ececb-0625-46a3-ad5f-3069c2da4048'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.931289 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76dd54851aa84efc8a947cdfe5e0c99f', 'request_id': 'req-256ececb-0625-46a3-ad5f-3069c2da4048'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.942691 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] The token cache is being invalidate because project 76dd54851aa84efc8a947cdfe5e0c99f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:34:11.942983 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.943732 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-256ececb-0625-46a3-ad5f-3069c2da4048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:11.945298 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1366/2721] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:34:11 2022] DELETE /identity/v3/projects/76dd54851aa84efc8a947cdfe5e0c99f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:34:11.950241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4811a3-0dd7-4607-a489-0c6de3969b2c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:11.950718 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4811a3-0dd7-4607-a489-0c6de3969b2c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:11.950816 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4811a3-0dd7-4607-a489-0c6de3969b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:11.968670 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-0e4811a3-0dd7-4607-a489-0c6de3969b2c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:34:11.969906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0e4811a3-0dd7-4607-a489-0c6de3969b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:12.027967 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1356/2722] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:34:12.036597 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cba023-e36c-4c14-bd31-14629af6ef2d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.036888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cba023-e36c-4c14-bd31-14629af6ef2d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.037165 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cba023-e36c-4c14-bd31-14629af6ef2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.055877 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-44cba023-e36c-4c14-bd31-14629af6ef2d None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:34:12.057062 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-44cba023-e36c-4c14-bd31-14629af6ef2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:12.064913 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1367/2723] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:34:12.069656 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4811a3-0dd7-4607-a489-0c6de3969b2c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:12.079053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4811a3-0dd7-4607-a489-0c6de3969b2c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:12.083883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:12.084802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.085083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.085359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.086120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:34:12.086424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:12.088030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:12.099367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1357/2724] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:34:12.103912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cba023-e36c-4c14-bd31-14629af6ef2d None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:12.113575 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cba023-e36c-4c14-bd31-14629af6ef2d None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:12.118324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a34f1-5feb-42ef-af05-16c3227dc546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:12.118971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a34f1-5feb-42ef-af05-16c3227dc546 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.119288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a34f1-5feb-42ef-af05-16c3227dc546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.119565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a34f1-5feb-42ef-af05-16c3227dc546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.120258 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a34f1-5feb-42ef-af05-16c3227dc546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:12.122189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a34f1-5feb-42ef-af05-16c3227dc546 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:12.205574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1368/2725] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:34:12 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:34:12.209895 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:12.219664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876175f6-74a7-406d-93d2-e052a0fbe770 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:12.273329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537be37-057f-4807-a518-d14f780015e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:12.274015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537be37-057f-4807-a518-d14f780015e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.274299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537be37-057f-4807-a518-d14f780015e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.274579 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537be37-057f-4807-a518-d14f780015e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.275248 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537be37-057f-4807-a518-d14f780015e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:12.277143 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537be37-057f-4807-a518-d14f780015e9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:12.371011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1358/2726] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:34:12 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:34:12.375369 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a34f1-5feb-42ef-af05-16c3227dc546 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:12.386080 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a34f1-5feb-42ef-af05-16c3227dc546 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:12.391143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6cd76-96dc-4c71-856d-24e79a25975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:12.391850 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6cd76-96dc-4c71-856d-24e79a25975e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.392132 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6cd76-96dc-4c71-856d-24e79a25975e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.392408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6cd76-96dc-4c71-856d-24e79a25975e admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.393131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6cd76-96dc-4c71-856d-24e79a25975e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:12.394763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6cd76-96dc-4c71-856d-24e79a25975e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:12.405447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1369/2727] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:34:12.410252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537be37-057f-4807-a518-d14f780015e9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:12.420503 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537be37-057f-4807-a518-d14f780015e9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:12.425974 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1797b-7c82-4955-950a-65338dc56bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:12.426758 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1797b-7c82-4955-950a-65338dc56bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.427049 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1797b-7c82-4955-950a-65338dc56bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.427339 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1797b-7c82-4955-950a-65338dc56bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.428061 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1797b-7c82-4955-950a-65338dc56bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:12.429655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1797b-7c82-4955-950a-65338dc56bdd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:12.439703 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1359/2728] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:34:12.443720 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6cd76-96dc-4c71-856d-24e79a25975e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:12.453490 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6cd76-96dc-4c71-856d-24e79a25975e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:12.458234 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:12.459295 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.459594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.459876 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] PATH_INFO: `/v3/projects/bee2f0ddaee64990ac823830d4d0aae0/users/116a288454e74b3c87800e2bb46dc9bf/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.474520 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee2f0ddaee64990ac823830d4d0aae0, user_id=116a288454e74b3c87800e2bb46dc9bf, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:12.477203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:12.558749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1370/2729] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:34:12 2022] PUT /identity/v3/projects/bee2f0ddaee64990ac823830d4d0aae0/users/116a288454e74b3c87800e2bb46dc9bf/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:34:12.569459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a7c0-8726-4748-a886-843e7cdf8bfb None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.569753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a7c0-8726-4748-a886-843e7cdf8bfb None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.570081 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a7c0-8726-4748-a886-843e7cdf8bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.596287 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d8f7a7c0-8726-4748-a886-843e7cdf8bfb None None] MFA Rules not processed for user `116a288454e74b3c87800e2bb46dc9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:34:12.597490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d8f7a7c0-8726-4748-a886-843e7cdf8bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:12.657332 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1360/2730] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:34:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:34:12.668288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:12.669991 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:12.724405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2891f-408d-471f-a881-59eaf8742205 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:12.728042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c7447-bc58-44e3-82c1-646ad99f9c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:12.728681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7447-bc58-44e3-82c1-646ad99f9c54 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.728967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7447-bc58-44e3-82c1-646ad99f9c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.729246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7447-bc58-44e3-82c1-646ad99f9c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.737287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c7447-bc58-44e3-82c1-646ad99f9c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:12.739277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c7447-bc58-44e3-82c1-646ad99f9c54 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:12.747808 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1371/2731] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:34:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:34:12.773654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7a7c0-8726-4748-a886-843e7cdf8bfb None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:12.832255 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7a7c0-8726-4748-a886-843e7cdf8bfb None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:12.837076 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:12.837960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:12.838241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:12.838516 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] PATH_INFO: `/v3/users/116a288454e74b3c87800e2bb46dc9bf` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:12.839695 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=116a288454e74b3c87800e2bb46dc9bf)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:12.841590 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:13.024209 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116a288454e74b3c87800e2bb46dc9bf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.024831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116a288454e74b3c87800e2bb46dc9bf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.025385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116a288454e74b3c87800e2bb46dc9bf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.031894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116a288454e74b3c87800e2bb46dc9bf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.103109 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116a288454e74b3c87800e2bb46dc9bf'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.119766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1361/2732] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:34:12 2022] DELETE /identity/v3/users/116a288454e74b3c87800e2bb46dc9bf => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:34:13.124274 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf7ba-0cd3-466b-a229-7af2b3e18e16 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:13.124573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf7ba-0cd3-466b-a229-7af2b3e18e16 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:13.124861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cf7ba-0cd3-466b-a229-7af2b3e18e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:13.141920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ba1cf7ba-0cd3-466b-a229-7af2b3e18e16 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:34:13.143121 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ba1cf7ba-0cd3-466b-a229-7af2b3e18e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:13.199340 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1372/2733] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:34:13.209574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:13.211030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:13.265992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb89eee-a479-42e0-97ce-43c32dbbba4d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:13.270220 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:13.270864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:13.271147 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:13.271424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:13.280416 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:13.281807 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:13.291088 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1362/2734] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:34:13.686184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cf7ba-0cd3-466b-a229-7af2b3e18e16 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:13.687642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ba1cf7ba-0cd3-466b-a229-7af2b3e18e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:13.700439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cf7ba-0cd3-466b-a229-7af2b3e18e16 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:13.705161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:13.706117 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:13.706901 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:13.707189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] PATH_INFO: `/v3/projects/bee2f0ddaee64990ac823830d4d0aae0` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:13.708415 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee2f0ddaee64990ac823830d4d0aae0)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:13.710642 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:13.810288 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee2f0ddaee64990ac823830d4d0aae0', 'request_id': 'req-ffd18a03-d01e-40af-9b59-71aba179f257'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.815368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee2f0ddaee64990ac823830d4d0aae0', 'request_id': 'req-ffd18a03-d01e-40af-9b59-71aba179f257'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.891624 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee2f0ddaee64990ac823830d4d0aae0', 'request_id': 'req-ffd18a03-d01e-40af-9b59-71aba179f257'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.904060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] The token cache is being invalidate because project bee2f0ddaee64990ac823830d4d0aae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:34:13.904351 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.904980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ffd18a03-d01e-40af-9b59-71aba179f257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:34:13.906554 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1373/2735] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:34:13 2022] DELETE /identity/v3/projects/bee2f0ddaee64990ac823830d4d0aae0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:34:27.984459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:27.985887 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:28.041249 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7240402-b228-4257-b0b4-a8e154bd1b1d service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:28.087285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:28.087933 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:28.088222 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:28.088504 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:28.090325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:28.145987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:28.147420 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:28.156177 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1363/2736] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:34:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:34:36.442689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482fed-74c2-4e9d-977d-e337b335e1a1 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:36.443223 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482fed-74c2-4e9d-977d-e337b335e1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:36.443330 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482fed-74c2-4e9d-977d-e337b335e1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:36.461997 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-90482fed-74c2-4e9d-977d-e337b335e1a1 None None] MFA Rules not processed for user `8101850dd18f4108ac12ff9487cbbe3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:34:36.463256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-90482fed-74c2-4e9d-977d-e337b335e1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:36.470158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1374/2737] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:34:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:34:53.895782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:34:53.897213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:53.953694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf65ed9-aefd-44e8-974e-bcad8ddf420e service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:34:53.960552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:34:53.960552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:34:53.960552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:34:53.960552 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:34:53.961937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:34:54.024153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:34:54.025521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:34:54.035203 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1364/2738] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:34:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:35:01.662466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15492-7cfc-42ea-b1e5-e9750a599b8e None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:01.665481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15492-7cfc-42ea-b1e5-e9750a599b8e None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:01.665481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15492-7cfc-42ea-b1e5-e9750a599b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:01.680804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-28f15492-7cfc-42ea-b1e5-e9750a599b8e None None] MFA Rules not processed for user `79dbde2a65224927af4bb5e77090496c`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:35:01.682043 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-28f15492-7cfc-42ea-b1e5-e9750a599b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:35:01.733668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1375/2739] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:35:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:35:12.752083 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:35:12.753509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:35:12.764490 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95506513-f0c3-40dd-bd83-816313f959e3 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:35:12.768140 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b87421-ed74-4a68-b012-4950f164739f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:35:12.768776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87421-ed74-4a68-b012-4950f164739f service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:12.769062 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87421-ed74-4a68-b012-4950f164739f service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:12.769340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87421-ed74-4a68-b012-4950f164739f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:12.771063 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-48b87421-ed74-4a68-b012-4950f164739f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:35:12.841923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b87421-ed74-4a68-b012-4950f164739f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:35:12.843358 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b87421-ed74-4a68-b012-4950f164739f service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:35:12.854640 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1365/2740] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:35:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:35:13.340604 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f15492-7cfc-42ea-b1e5-e9750a599b8e None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:35:13.350155 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f15492-7cfc-42ea-b1e5-e9750a599b8e None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:35:13.353662 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce484b-8e50-48ee-a88c-2186978c45b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:35:13.354331 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce484b-8e50-48ee-a88c-2186978c45b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:13.354614 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce484b-8e50-48ee-a88c-2186978c45b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:13.354889 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce484b-8e50-48ee-a88c-2186978c45b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:13.356656 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-48ce484b-8e50-48ee-a88c-2186978c45b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:35:13.369476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce484b-8e50-48ee-a88c-2186978c45b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:35:13.370946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce484b-8e50-48ee-a88c-2186978c45b9 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:35:13.381560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1376/2741] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:35:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:35:17.418677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64eddd-9684-4e06-833c-27153c487587 None None] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:17.419583 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64eddd-9684-4e06-833c-27153c487587 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:17.420037 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64eddd-9684-4e06-833c-27153c487587 None None] PATH_INFO: `/` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:17.422059 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1366/2742] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:35:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 13 13:35:17.433271 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a1ab3-0d46-4632-8297-ef0bb60c304d None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:17.434183 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a1ab3-0d46-4632-8297-ef0bb60c304d None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:17.434472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a1ab3-0d46-4632-8297-ef0bb60c304d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:17.482091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-5b0a1ab3-0d46-4632-8297-ef0bb60c304d None None] MFA Rules not processed for user `8ddb0dcaa5304016af1bc1f88845fb80`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:35:17.483456 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-5b0a1ab3-0d46-4632-8297-ef0bb60c304d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:35:17.504874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1377/2743] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:35:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:35:17.518438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64eddd-9684-4e06-833c-27153c487587 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:35:17.531426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64eddd-9684-4e06-833c-27153c487587 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:35:17.535840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67cd71-fd45-4cd1-a318-30b15421f29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:35:17.536712 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67cd71-fd45-4cd1-a318-30b15421f29f service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:17.536994 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67cd71-fd45-4cd1-a318-30b15421f29f service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:17.537271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67cd71-fd45-4cd1-a318-30b15421f29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:17.545124 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67cd71-fd45-4cd1-a318-30b15421f29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:35:17.546685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67cd71-fd45-4cd1-a318-30b15421f29f service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:35:17.555796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1367/2744] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:35:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:35:17.643733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2167da-a00b-4a89-ada0-5c6dedebd58f None None] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:17.644027 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2167da-a00b-4a89-ada0-5c6dedebd58f None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:17.644309 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2167da-a00b-4a89-ada0-5c6dedebd58f None None] PATH_INFO: `/` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:17.645724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1378/2745] 10.222.0.25 () {54 vars in 921 bytes} [Fri May 13 13:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 13 13:35:17.653953 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dda9f-e981-40f7-a5f2-a76bd691ec69 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:17.654257 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dda9f-e981-40f7-a5f2-a76bd691ec69 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:17.654513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dda9f-e981-40f7-a5f2-a76bd691ec69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:17.677582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-751dda9f-e981-40f7-a5f2-a76bd691ec69 None None] MFA Rules not processed for user `3f94e52e32324d8cbf38fb05dc38ea96`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:35:17.678894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-751dda9f-e981-40f7-a5f2-a76bd691ec69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:35:17.689284 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1368/2746] 10.222.0.25 () {58 vars in 1018 bytes} [Fri May 13 13:35:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:35:17.701477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2167da-a00b-4a89-ada0-5c6dedebd58f None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:35:17.713344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2167da-a00b-4a89-ada0-5c6dedebd58f None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:35:17.717045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:35:17.717696 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:35:17.718057 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:35:17.718345 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:35:17.729637 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:35:17.731149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:35:17.742221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1379/2747] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:35:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:36:04.414264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dda9f-e981-40f7-a5f2-a76bd691ec69 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:04.416463 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-751dda9f-e981-40f7-a5f2-a76bd691ec69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:04.430532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dda9f-e981-40f7-a5f2-a76bd691ec69 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:04.436733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:04.437401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:04.437684 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:04.437992 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:04.440329 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:04.457918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:04.457918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:04.467844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1369/2748] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:09.875016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:09.876413 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:09.888146 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42428491-255a-4871-8c3a-d4a80e21de26 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:09.891972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:09.892645 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:09.892933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:09.893215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:09.894992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:09.906166 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:09.907592 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:09.916368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1380/2749] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:11.417161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:11.426880 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5333b9-654c-4864-82ac-c8671308005b service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:11.430535 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:11.431160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:11.431438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:11.431710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:11.433533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:11.494592 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:11.496622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:11.506105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1370/2750] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:15.229218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:15.231835 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:15.246851 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a1e2b-6da1-4247-9fb6-4561d2b686a7 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:15.251517 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:15.252394 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:15.253063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:15.253063 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] PATH_INFO: `/v3/users/79dbde2a65224927af4bb5e77090496c` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:15.254209 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79dbde2a65224927af4bb5e77090496c)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:15.256136 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:15.409270 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79dbde2a65224927af4bb5e77090496c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:15.409883 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79dbde2a65224927af4bb5e77090496c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:15.482399 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79dbde2a65224927af4bb5e77090496c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:15.491368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79dbde2a65224927af4bb5e77090496c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:15.492004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-aa1a366a-6eb7-4f6f-ac7f-f35414b06b8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79dbde2a65224927af4bb5e77090496c'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:15.509254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1381/2751] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:36:15 2022] DELETE /identity/v3/users/79dbde2a65224927af4bb5e77090496c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:15.754875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:15.766708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7044313-c80f-4644-8a47-5c98e4912077 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:15.821798 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:15.822656 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:15.822938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:15.823211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] PATH_INFO: `/v3/projects/1e72f39546704606927bcefdc0f13235` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:15.825030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e72f39546704606927bcefdc0f13235)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:15.826969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:15.942308 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e72f39546704606927bcefdc0f13235', 'request_id': 'req-4966d284-213c-4b34-951e-95618dcc92dd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:15.947486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e72f39546704606927bcefdc0f13235', 'request_id': 'req-4966d284-213c-4b34-951e-95618dcc92dd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:16.005178 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e72f39546704606927bcefdc0f13235', 'request_id': 'req-4966d284-213c-4b34-951e-95618dcc92dd'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:16.016785 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] The token cache is being invalidate because project 1e72f39546704606927bcefdc0f13235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:36:16.017071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:16.017701 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4966d284-213c-4b34-951e-95618dcc92dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:16.019261 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1371/2752] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:36:15 2022] DELETE /identity/v3/projects/1e72f39546704606927bcefdc0f13235 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:16.024174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219236-43cd-4c6f-9b64-e5bdd24cbb32 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.024475 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219236-43cd-4c6f-9b64-e5bdd24cbb32 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.024762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219236-43cd-4c6f-9b64-e5bdd24cbb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.043610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-75219236-43cd-4c6f-9b64-e5bdd24cbb32 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:16.045203 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-75219236-43cd-4c6f-9b64-e5bdd24cbb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:16.102381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1382/2753] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:16.110817 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1bc50-5aea-49b6-bc36-ed3cf89f36db None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.111105 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1bc50-5aea-49b6-bc36-ed3cf89f36db None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.111387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1bc50-5aea-49b6-bc36-ed3cf89f36db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.129153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-cbc1bc50-5aea-49b6-bc36-ed3cf89f36db None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:16.130392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-cbc1bc50-5aea-49b6-bc36-ed3cf89f36db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:16.138861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1372/2754] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:16.142986 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75219236-43cd-4c6f-9b64-e5bdd24cbb32 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:16.152984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75219236-43cd-4c6f-9b64-e5bdd24cbb32 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:16.157574 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:16.158266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.158573 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.158837 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.159563 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:36:16.159874 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:16.161504 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:16.172702 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1383/2755] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:36:16.177099 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1bc50-5aea-49b6-bc36-ed3cf89f36db None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:16.186186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1bc50-5aea-49b6-bc36-ed3cf89f36db None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:16.190829 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe2cad-98b7-4031-b7eb-f7ebb563ac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:16.191472 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe2cad-98b7-4031-b7eb-f7ebb563ac22 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.191755 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe2cad-98b7-4031-b7eb-f7ebb563ac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.192030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe2cad-98b7-4031-b7eb-f7ebb563ac22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.192714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe2cad-98b7-4031-b7eb-f7ebb563ac22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:16.194659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe2cad-98b7-4031-b7eb-f7ebb563ac22 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:16.315797 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1373/2756] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:36:16 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:16.319933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:16.330134 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe37465-7f0f-4c23-85b0-bf4746a2f3f1 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:16.384438 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91b0a2-233c-419d-91c4-a06e91973aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:16.385104 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91b0a2-233c-419d-91c4-a06e91973aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.385393 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91b0a2-233c-419d-91c4-a06e91973aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.385674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91b0a2-233c-419d-91c4-a06e91973aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.386485 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91b0a2-233c-419d-91c4-a06e91973aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:16.388411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91b0a2-233c-419d-91c4-a06e91973aa0 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:16.514139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1384/2757] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:36:16 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:16.519889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe2cad-98b7-4031-b7eb-f7ebb563ac22 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:16.529291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe2cad-98b7-4031-b7eb-f7ebb563ac22 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:16.534028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2bea8-fdb7-4ca1-a133-e6392865fa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:16.534689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2bea8-fdb7-4ca1-a133-e6392865fa39 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.534969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2bea8-fdb7-4ca1-a133-e6392865fa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.535243 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2bea8-fdb7-4ca1-a133-e6392865fa39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.535963 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2bea8-fdb7-4ca1-a133-e6392865fa39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:16.537588 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2bea8-fdb7-4ca1-a133-e6392865fa39 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:16.548757 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1374/2758] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:36:16.552794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91b0a2-233c-419d-91c4-a06e91973aa0 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:16.562888 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91b0a2-233c-419d-91c4-a06e91973aa0 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:16.567553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c8965-fd61-4da0-bf99-d9aa1ca8df21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:16.568217 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c8965-fd61-4da0-bf99-d9aa1ca8df21 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.568495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c8965-fd61-4da0-bf99-d9aa1ca8df21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.568769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c8965-fd61-4da0-bf99-d9aa1ca8df21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.569487 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c8965-fd61-4da0-bf99-d9aa1ca8df21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:16.571182 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c8965-fd61-4da0-bf99-d9aa1ca8df21 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:16.581006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1385/2759] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:36:16.585173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2bea8-fdb7-4ca1-a133-e6392865fa39 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:16.596442 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2bea8-fdb7-4ca1-a133-e6392865fa39 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:16.601580 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8f15f-8412-4a8c-b1df-d550ad5bc584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:16.602569 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f15f-8412-4a8c-b1df-d550ad5bc584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.602859 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f15f-8412-4a8c-b1df-d550ad5bc584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.603141 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f15f-8412-4a8c-b1df-d550ad5bc584 admin admin] PATH_INFO: `/v3/projects/5f0b24506ab5420e8c9803cc2454b400/users/72488fa1846645399d89baf59e0722ff/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.618478 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8f15f-8412-4a8c-b1df-d550ad5bc584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0b24506ab5420e8c9803cc2454b400, user_id=72488fa1846645399d89baf59e0722ff, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:16.621158 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8f15f-8412-4a8c-b1df-d550ad5bc584 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:16.711495 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1375/2760] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:16 2022] PUT /identity/v3/projects/5f0b24506ab5420e8c9803cc2454b400/users/72488fa1846645399d89baf59e0722ff/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:36:16.722249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef3cba-d958-4092-8ee4-cf9530041d09 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.722548 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef3cba-d958-4092-8ee4-cf9530041d09 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.722834 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef3cba-d958-4092-8ee4-cf9530041d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.748927 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-00ef3cba-d958-4092-8ee4-cf9530041d09 None None] MFA Rules not processed for user `72488fa1846645399d89baf59e0722ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:16.750318 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-00ef3cba-d958-4092-8ee4-cf9530041d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:16.810944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1386/2761] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:16.815386 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8f15f-8412-4a8c-b1df-d550ad5bc584 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:16.877708 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8f15f-8412-4a8c-b1df-d550ad5bc584 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:16.882256 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0a600-363e-42e0-824e-346ed7c23ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:16.882910 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a600-363e-42e0-824e-346ed7c23ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:16.883194 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a600-363e-42e0-824e-346ed7c23ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:16.883475 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a600-363e-42e0-824e-346ed7c23ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:16.884164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0a600-363e-42e0-824e-346ed7c23ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:16.886669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0a600-363e-42e0-824e-346ed7c23ba3 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:16.974778 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1376/2762] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:36:16 2022] POST /identity/v3/projects => generated 363 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:16.978822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef3cba-d958-4092-8ee4-cf9530041d09 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:16.989368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef3cba-d958-4092-8ee4-cf9530041d09 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:17.043481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349a64d-11c0-476d-84f2-b74419d542f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:17.044161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349a64d-11c0-476d-84f2-b74419d542f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:17.044450 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349a64d-11c0-476d-84f2-b74419d542f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:17.044729 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349a64d-11c0-476d-84f2-b74419d542f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:17.045410 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349a64d-11c0-476d-84f2-b74419d542f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:17.047499 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349a64d-11c0-476d-84f2-b74419d542f6 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:17.122221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1387/2763] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:36:16 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:17.126182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0a600-363e-42e0-824e-346ed7c23ba3 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:17.136966 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0a600-363e-42e0-824e-346ed7c23ba3 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:17.141533 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23e651-99d5-4821-bafc-affaff6e3345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:17.142216 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23e651-99d5-4821-bafc-affaff6e3345 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:17.142501 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23e651-99d5-4821-bafc-affaff6e3345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:17.142776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23e651-99d5-4821-bafc-affaff6e3345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:17.143526 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23e651-99d5-4821-bafc-affaff6e3345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:17.145121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23e651-99d5-4821-bafc-affaff6e3345 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:17.155481 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1377/2764] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:17.159459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349a64d-11c0-476d-84f2-b74419d542f6 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:17.169436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349a64d-11c0-476d-84f2-b74419d542f6 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:17.174139 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d91e63-30a6-4dec-9b0c-2a2686d50e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:17.175007 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d91e63-30a6-4dec-9b0c-2a2686d50e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:17.175287 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d91e63-30a6-4dec-9b0c-2a2686d50e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:17.175565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d91e63-30a6-4dec-9b0c-2a2686d50e74 admin admin] PATH_INFO: `/v3/projects/6d521a47d9bd401c9cfdc6fb5add02cf/users/632774d095c64926be62a22150223872/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:17.190762 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d91e63-30a6-4dec-9b0c-2a2686d50e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d521a47d9bd401c9cfdc6fb5add02cf, user_id=632774d095c64926be62a22150223872, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:17.193424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d91e63-30a6-4dec-9b0c-2a2686d50e74 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:17.288314 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1388/2765] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:17 2022] PUT /identity/v3/projects/6d521a47d9bd401c9cfdc6fb5add02cf/users/632774d095c64926be62a22150223872/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:17.292952 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23e651-99d5-4821-bafc-affaff6e3345 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:17.303055 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23e651-99d5-4821-bafc-affaff6e3345 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:17.357114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9622e5-89cc-4e60-bb68-9b530ec850b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:17.357771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9622e5-89cc-4e60-bb68-9b530ec850b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:17.358160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9622e5-89cc-4e60-bb68-9b530ec850b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:17.358443 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9622e5-89cc-4e60-bb68-9b530ec850b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:17.359159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9622e5-89cc-4e60-bb68-9b530ec850b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:17.360764 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9622e5-89cc-4e60-bb68-9b530ec850b7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:17.370218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1378/2766] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:17.374281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d91e63-30a6-4dec-9b0c-2a2686d50e74 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:17.384421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d91e63-30a6-4dec-9b0c-2a2686d50e74 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:17.389093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:17.389951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:17.390237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:17.390515 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] PATH_INFO: `/v3/projects/6d521a47d9bd401c9cfdc6fb5add02cf/users/632774d095c64926be62a22150223872/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:17.392482 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d521a47d9bd401c9cfdc6fb5add02cf, user_id=632774d095c64926be62a22150223872, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:17.395161 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:17.405447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1389/2767] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:17 2022] PUT /identity/v3/projects/6d521a47d9bd401c9cfdc6fb5add02cf/users/632774d095c64926be62a22150223872/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:17.416663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072c549-c29d-45df-9b33-beee7604205d None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:17.418288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072c549-c29d-45df-9b33-beee7604205d None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:17.418288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072c549-c29d-45df-9b33-beee7604205d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:17.444169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-4072c549-c29d-45df-9b33-beee7604205d None None] MFA Rules not processed for user `632774d095c64926be62a22150223872`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:17.445450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-4072c549-c29d-45df-9b33-beee7604205d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:17.512537 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1379/2768] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:17.523769 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:17.525478 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:17.580355 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbba10-9e78-49d1-baae-ce715d2b1b08 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:17.584095 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62ac4a-5226-4dc8-aba8-3815513e4c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:17.584734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ac4a-5226-4dc8-aba8-3815513e4c6c service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:17.585022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ac4a-5226-4dc8-aba8-3815513e4c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:17.585303 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62ac4a-5226-4dc8-aba8-3815513e4c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:17.594372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62ac4a-5226-4dc8-aba8-3815513e4c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:17.595804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62ac4a-5226-4dc8-aba8-3815513e4c6c service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:17.605516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1390/2769] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:18.502375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072c549-c29d-45df-9b33-beee7604205d None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:18.565026 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072c549-c29d-45df-9b33-beee7604205d None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:18.569548 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:18.570438 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:18.570726 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:18.571007 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] PATH_INFO: `/v3/users/72488fa1846645399d89baf59e0722ff` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:18.572202 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72488fa1846645399d89baf59e0722ff)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:18.574725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:18.838796 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72488fa1846645399d89baf59e0722ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:18.839385 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72488fa1846645399d89baf59e0722ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:18.839993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72488fa1846645399d89baf59e0722ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:18.846696 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72488fa1846645399d89baf59e0722ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:18.958782 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72488fa1846645399d89baf59e0722ff'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:18.978851 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1380/2770] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:36:18 2022] DELETE /identity/v3/users/72488fa1846645399d89baf59e0722ff => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:18.983939 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce7a5-99e9-437d-b4e0-507117b78777 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:18.984448 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce7a5-99e9-437d-b4e0-507117b78777 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:18.984547 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce7a5-99e9-437d-b4e0-507117b78777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:19.002559 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-a00ce7a5-99e9-437d-b4e0-507117b78777 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:19.003764 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a00ce7a5-99e9-437d-b4e0-507117b78777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:19.059974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1391/2771] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:19.069721 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:19.071152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:19.126250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7ee82-7a75-4a86-ab64-491b5ec1b860 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:19.129761 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3862250-1b33-4254-8f08-dba65a55f550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:19.130426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3862250-1b33-4254-8f08-dba65a55f550 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:19.130710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3862250-1b33-4254-8f08-dba65a55f550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:19.130997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3862250-1b33-4254-8f08-dba65a55f550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:19.140182 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3862250-1b33-4254-8f08-dba65a55f550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:19.141609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3862250-1b33-4254-8f08-dba65a55f550 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:19.151015 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1381/2772] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:19.612341 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce7a5-99e9-437d-b4e0-507117b78777 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:19.618367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a00ce7a5-99e9-437d-b4e0-507117b78777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:19.626955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce7a5-99e9-437d-b4e0-507117b78777 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:19.632048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:19.632912 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:19.633211 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:19.633463 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] PATH_INFO: `/v3/projects/5f0b24506ab5420e8c9803cc2454b400` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:19.634690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0b24506ab5420e8c9803cc2454b400)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:19.637510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:19.738582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0b24506ab5420e8c9803cc2454b400', 'request_id': 'req-06648dc8-a98d-46d9-9456-ae69257246d1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:19.744231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0b24506ab5420e8c9803cc2454b400', 'request_id': 'req-06648dc8-a98d-46d9-9456-ae69257246d1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:19.821919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0b24506ab5420e8c9803cc2454b400', 'request_id': 'req-06648dc8-a98d-46d9-9456-ae69257246d1'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:19.832670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] The token cache is being invalidate because project 5f0b24506ab5420e8c9803cc2454b400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:36:19.832954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:19.833565 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:19.835153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1392/2773] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:36:19 2022] DELETE /identity/v3/projects/5f0b24506ab5420e8c9803cc2454b400 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:19.839439 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3862250-1b33-4254-8f08-dba65a55f550 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:19.848877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3862250-1b33-4254-8f08-dba65a55f550 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:19.901649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:19.902539 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:19.902819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:19.903093 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] PATH_INFO: `/v3/users/632774d095c64926be62a22150223872` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:19.904264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=632774d095c64926be62a22150223872)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:19.906228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:20.104559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '632774d095c64926be62a22150223872'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.105717 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '632774d095c64926be62a22150223872'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.106367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '632774d095c64926be62a22150223872'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.113139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '632774d095c64926be62a22150223872'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.168377 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '632774d095c64926be62a22150223872'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.185327 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1382/2774] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:36:19 2022] DELETE /identity/v3/users/632774d095c64926be62a22150223872 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:20.590062 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:20.593112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:20.605149 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648dc8-a98d-46d9-9456-ae69257246d1 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:20.660646 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:20.661514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:20.661818 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:20.662124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] PATH_INFO: `/v3/projects/6d521a47d9bd401c9cfdc6fb5add02cf` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:20.663299 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d521a47d9bd401c9cfdc6fb5add02cf)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:20.665214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:20.781562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d521a47d9bd401c9cfdc6fb5add02cf', 'request_id': 'req-ba216749-89fd-47e9-a3ae-62dc99f27487'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.786733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d521a47d9bd401c9cfdc6fb5add02cf', 'request_id': 'req-ba216749-89fd-47e9-a3ae-62dc99f27487'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.826870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:20.828297 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:20.853355 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d521a47d9bd401c9cfdc6fb5add02cf', 'request_id': 'req-ba216749-89fd-47e9-a3ae-62dc99f27487'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.864955 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] The token cache is being invalidate because project 6d521a47d9bd401c9cfdc6fb5add02cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:36:20.865198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.865879 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-ba216749-89fd-47e9-a3ae-62dc99f27487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:20.867449 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1393/2775] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:36:20 2022] DELETE /identity/v3/projects/6d521a47d9bd401c9cfdc6fb5add02cf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:20.873008 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cf11c-0706-455f-8197-61eef33847ce None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:20.873300 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cf11c-0706-455f-8197-61eef33847ce None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:20.873580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cf11c-0706-455f-8197-61eef33847ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:20.888984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1099-eadd-4278-ac34-9ece41560561 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:20.891841 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-0a5cf11c-0706-455f-8197-61eef33847ce None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:20.893022 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-0a5cf11c-0706-455f-8197-61eef33847ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:20.935744 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:20.936395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:20.936676 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:20.936950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:20.938748 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:20.947873 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:20.949875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:20.950204 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1394/2776] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:36:20.957610 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e25da-389f-425a-ab60-e0becb694145 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:20.957930 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e25da-389f-425a-ab60-e0becb694145 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:20.958208 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e25da-389f-425a-ab60-e0becb694145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:20.959075 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1383/2777] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 13 13:36:20.976670 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-6e8e25da-389f-425a-ab60-e0becb694145 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:20.977899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-6e8e25da-389f-425a-ab60-e0becb694145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:20.985715 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1395/2778] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:36:20.989976 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:21.001435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d340ae5-c08f-4e0b-bad0-a3dc4896ecb1 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:21.006046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:21.006724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.007022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.007333 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.008064 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:36:21.008376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:21.010012 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:21.021753 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1384/2779] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:36:21.025895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e25da-389f-425a-ab60-e0becb694145 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:21.035684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e25da-389f-425a-ab60-e0becb694145 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:21.040436 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dc3cb-0c80-462e-b434-95816ea9ee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:21.041107 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dc3cb-0c80-462e-b434-95816ea9ee66 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.041390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dc3cb-0c80-462e-b434-95816ea9ee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.041663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dc3cb-0c80-462e-b434-95816ea9ee66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.042461 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dc3cb-0c80-462e-b434-95816ea9ee66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:21.044381 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dc3cb-0c80-462e-b434-95816ea9ee66 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:21.141688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1396/2780] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:36:21 2022] POST /identity/v3/projects => generated 355 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:21.142763 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:21.152678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1139d0-a3da-43b6-93ee-c34809eb32ed admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:21.208250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c009f-9c7d-4cd9-9023-00c8e0bfbf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:21.208931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c009f-9c7d-4cd9-9023-00c8e0bfbf8d admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.209213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c009f-9c7d-4cd9-9023-00c8e0bfbf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.209486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c009f-9c7d-4cd9-9023-00c8e0bfbf8d admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.210218 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c009f-9c7d-4cd9-9023-00c8e0bfbf8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:21.212431 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c009f-9c7d-4cd9-9023-00c8e0bfbf8d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:21.310858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1385/2781] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:36:21 2022] POST /identity/v3/users => generated 278 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:36:21.314919 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dc3cb-0c80-462e-b434-95816ea9ee66 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:21.326191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dc3cb-0c80-462e-b434-95816ea9ee66 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:21.330899 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63725fb7-5562-4932-90e0-87aa08e6449e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:21.331568 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63725fb7-5562-4932-90e0-87aa08e6449e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.331854 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63725fb7-5562-4932-90e0-87aa08e6449e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.332131 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63725fb7-5562-4932-90e0-87aa08e6449e admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.332855 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63725fb7-5562-4932-90e0-87aa08e6449e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:21.334518 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63725fb7-5562-4932-90e0-87aa08e6449e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:21.346206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1397/2782] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:21.350125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c009f-9c7d-4cd9-9023-00c8e0bfbf8d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:21.360978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c009f-9c7d-4cd9-9023-00c8e0bfbf8d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:21.365725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e59b6c-158e-4dff-a788-09f94506380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:21.366424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e59b6c-158e-4dff-a788-09f94506380a admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.366710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e59b6c-158e-4dff-a788-09f94506380a admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.366991 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e59b6c-158e-4dff-a788-09f94506380a admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.367726 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e59b6c-158e-4dff-a788-09f94506380a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:21.369325 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e59b6c-158e-4dff-a788-09f94506380a admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:21.380754 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1386/2783] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:21.384472 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63725fb7-5562-4932-90e0-87aa08e6449e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:21.395745 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63725fb7-5562-4932-90e0-87aa08e6449e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:21.400352 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deb0c1-4e4d-4a24-981d-425a74311105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:21.401237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deb0c1-4e4d-4a24-981d-425a74311105 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.401555 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deb0c1-4e4d-4a24-981d-425a74311105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.402308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deb0c1-4e4d-4a24-981d-425a74311105 admin admin] PATH_INFO: `/v3/projects/d68146e14c8d42cc9650eab3e1233f73/users/d3b43390d46547c9a8b8d2c8d4633685/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.416221 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deb0c1-4e4d-4a24-981d-425a74311105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68146e14c8d42cc9650eab3e1233f73, user_id=d3b43390d46547c9a8b8d2c8d4633685, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:21.419546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deb0c1-4e4d-4a24-981d-425a74311105 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:21.534723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1398/2784] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:21 2022] PUT /identity/v3/projects/d68146e14c8d42cc9650eab3e1233f73/users/d3b43390d46547c9a8b8d2c8d4633685/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:21.545046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bc97c-d36d-4dcd-8a9d-c6567d36f6ff None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.545342 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bc97c-d36d-4dcd-8a9d-c6567d36f6ff None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.545646 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bc97c-d36d-4dcd-8a9d-c6567d36f6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.572285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-d08bc97c-d36d-4dcd-8a9d-c6567d36f6ff None None] MFA Rules not processed for user `d3b43390d46547c9a8b8d2c8d4633685`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:21.574159 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d08bc97c-d36d-4dcd-8a9d-c6567d36f6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:21.636068 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1387/2785] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:21 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:21.641160 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deb0c1-4e4d-4a24-981d-425a74311105 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:21.701066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deb0c1-4e4d-4a24-981d-425a74311105 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:21.705627 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e94f8c-4475-4287-b09d-7fc4f80e3be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:21.706305 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e94f8c-4475-4287-b09d-7fc4f80e3be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.706590 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e94f8c-4475-4287-b09d-7fc4f80e3be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.706868 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e94f8c-4475-4287-b09d-7fc4f80e3be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.707549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e94f8c-4475-4287-b09d-7fc4f80e3be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:21.709470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e94f8c-4475-4287-b09d-7fc4f80e3be1 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:21.832909 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1399/2786] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:36:21 2022] POST /identity/v3/projects => generated 355 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:21.837315 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bc97c-d36d-4dcd-8a9d-c6567d36f6ff None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:21.848079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bc97c-d36d-4dcd-8a9d-c6567d36f6ff None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:21.903954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b691-3632-4a12-be62-1ba95fed39bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:21.904631 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b691-3632-4a12-be62-1ba95fed39bb admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:21.904888 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b691-3632-4a12-be62-1ba95fed39bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:21.905164 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b691-3632-4a12-be62-1ba95fed39bb admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:21.905878 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83b691-3632-4a12-be62-1ba95fed39bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:21.907795 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83b691-3632-4a12-be62-1ba95fed39bb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:21.996527 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1388/2787] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:36:21 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:22.007697 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e94f8c-4475-4287-b09d-7fc4f80e3be1 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:22.010735 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e94f8c-4475-4287-b09d-7fc4f80e3be1 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:22.016277 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fb656-763e-4ff1-b9cf-c0c19d3d35bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:22.016945 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fb656-763e-4ff1-b9cf-c0c19d3d35bf admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:22.017235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fb656-763e-4ff1-b9cf-c0c19d3d35bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:22.017516 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fb656-763e-4ff1-b9cf-c0c19d3d35bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:22.018285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fb656-763e-4ff1-b9cf-c0c19d3d35bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:22.020619 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fb656-763e-4ff1-b9cf-c0c19d3d35bf admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:22.031875 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1400/2788] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:22.036145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b691-3632-4a12-be62-1ba95fed39bb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:22.046509 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b691-3632-4a12-be62-1ba95fed39bb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:22.051024 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed6dff-7ee9-46f5-aa3f-ab3a70fb8aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:22.051894 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed6dff-7ee9-46f5-aa3f-ab3a70fb8aff admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:22.052180 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed6dff-7ee9-46f5-aa3f-ab3a70fb8aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:22.052458 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed6dff-7ee9-46f5-aa3f-ab3a70fb8aff admin admin] PATH_INFO: `/v3/projects/2754d8ba11834ab4a55373bd0d97ea12/users/9281156219c64c87926834f5d8df6c3a/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:22.067086 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed6dff-7ee9-46f5-aa3f-ab3a70fb8aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2754d8ba11834ab4a55373bd0d97ea12, user_id=9281156219c64c87926834f5d8df6c3a, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:22.069734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed6dff-7ee9-46f5-aa3f-ab3a70fb8aff admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:22.116534 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1389/2789] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:22 2022] PUT /identity/v3/projects/2754d8ba11834ab4a55373bd0d97ea12/users/9281156219c64c87926834f5d8df6c3a/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:22.121411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fb656-763e-4ff1-b9cf-c0c19d3d35bf admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:22.131754 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fb656-763e-4ff1-b9cf-c0c19d3d35bf admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:22.185510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79e612-a827-451a-9bba-8e0f194c880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:22.186186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79e612-a827-451a-9bba-8e0f194c880f admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:22.186468 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79e612-a827-451a-9bba-8e0f194c880f admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:22.186744 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79e612-a827-451a-9bba-8e0f194c880f admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:22.187457 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79e612-a827-451a-9bba-8e0f194c880f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:22.189041 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79e612-a827-451a-9bba-8e0f194c880f admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:22.198285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1401/2790] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:22.203120 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed6dff-7ee9-46f5-aa3f-ab3a70fb8aff admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:22.213299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed6dff-7ee9-46f5-aa3f-ab3a70fb8aff admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:22.218576 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:22.219459 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:22.219737 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:22.220011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] PATH_INFO: `/v3/projects/2754d8ba11834ab4a55373bd0d97ea12/users/9281156219c64c87926834f5d8df6c3a/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:22.222039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2754d8ba11834ab4a55373bd0d97ea12, user_id=9281156219c64c87926834f5d8df6c3a, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:22.224649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:22.234394 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1390/2791] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:22 2022] PUT /identity/v3/projects/2754d8ba11834ab4a55373bd0d97ea12/users/9281156219c64c87926834f5d8df6c3a/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:22.245546 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7773e-b04a-4f68-8820-2f168c737151 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:22.251659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7773e-b04a-4f68-8820-2f168c737151 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:22.251659 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7773e-b04a-4f68-8820-2f168c737151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:22.272486 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-8bc7773e-b04a-4f68-8820-2f168c737151 None None] MFA Rules not processed for user `9281156219c64c87926834f5d8df6c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:22.273669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8bc7773e-b04a-4f68-8820-2f168c737151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:22.342558 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1402/2792] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:22.353687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:22.355423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:22.411397 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4089f4-c1fa-4bf7-87b8-843e3ed1d7ff admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:22.415058 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:22.415725 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:22.416011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:22.416290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:22.425067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:22.426574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:22.436328 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1391/2793] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:22 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:22.637970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7773e-b04a-4f68-8820-2f168c737151 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:22.646382 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7773e-b04a-4f68-8820-2f168c737151 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:22.650025 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50999d0-d59a-4698-b46f-4360f1f89145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:22.650663 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50999d0-d59a-4698-b46f-4360f1f89145 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:22.650953 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50999d0-d59a-4698-b46f-4360f1f89145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:22.651231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50999d0-d59a-4698-b46f-4360f1f89145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:22.707158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50999d0-d59a-4698-b46f-4360f1f89145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:22.708595 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50999d0-d59a-4698-b46f-4360f1f89145 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:22.718184 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1403/2794] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:22 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:39.290783 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:39.296314 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:39.348223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670fc0d-ce0a-4af3-bdb8-21b9003ac73b service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:39.351783 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7603023-abca-45e0-b580-e90789aaaaa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:39.352544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7603023-abca-45e0-b580-e90789aaaaa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:39.352827 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7603023-abca-45e0-b580-e90789aaaaa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:39.353108 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7603023-abca-45e0-b580-e90789aaaaa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:39.355411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-c7603023-abca-45e0-b580-e90789aaaaa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:39.418497 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7603023-abca-45e0-b580-e90789aaaaa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:39.419954 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7603023-abca-45e0-b580-e90789aaaaa0 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:39.429514 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1392/2795] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:40.620602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50999d0-d59a-4698-b46f-4360f1f89145 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:40.633465 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50999d0-d59a-4698-b46f-4360f1f89145 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:40.641145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:40.642454 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:40.643564 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:40.644031 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] PATH_INFO: `/v3/users/d3b43390d46547c9a8b8d2c8d4633685` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:40.646006 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b43390d46547c9a8b8d2c8d4633685)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:40.648678 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:40.845383 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b43390d46547c9a8b8d2c8d4633685'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:40.846000 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b43390d46547c9a8b8d2c8d4633685'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:40.939455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b43390d46547c9a8b8d2c8d4633685'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:40.947430 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b43390d46547c9a8b8d2c8d4633685'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:40.948426 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b43390d46547c9a8b8d2c8d4633685'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:40.969304 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1404/2796] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:36:40 2022] DELETE /identity/v3/users/d3b43390d46547c9a8b8d2c8d4633685 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:40.974828 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eee787-8fd4-4f31-a38c-002b4e5b8f49 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:40.975399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eee787-8fd4-4f31-a38c-002b4e5b8f49 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:40.975543 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eee787-8fd4-4f31-a38c-002b4e5b8f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:40.996799 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-22eee787-8fd4-4f31-a38c-002b4e5b8f49 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:40.998173 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-22eee787-8fd4-4f31-a38c-002b4e5b8f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:41.062677 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1393/2797] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:41.074706 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:41.076237 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:41.143770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d32ec-f668-4581-bb96-0696ed53ca6e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:41.147820 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cefcff-d8c2-4ae4-9cbe-dc476938102c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:41.148621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefcff-d8c2-4ae4-9cbe-dc476938102c service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:41.149005 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefcff-d8c2-4ae4-9cbe-dc476938102c service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:41.149380 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefcff-d8c2-4ae4-9cbe-dc476938102c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:41.159979 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cefcff-d8c2-4ae4-9cbe-dc476938102c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:41.161593 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cefcff-d8c2-4ae4-9cbe-dc476938102c service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:41.173164 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1405/2798] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:41.603886 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eee787-8fd4-4f31-a38c-002b4e5b8f49 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:41.605999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-22eee787-8fd4-4f31-a38c-002b4e5b8f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:41.619225 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eee787-8fd4-4f31-a38c-002b4e5b8f49 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:41.626430 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:41.627469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:41.627855 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:41.628234 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] PATH_INFO: `/v3/projects/d68146e14c8d42cc9650eab3e1233f73` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:41.630272 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d68146e14c8d42cc9650eab3e1233f73)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:41.632457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:41.749457 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd68146e14c8d42cc9650eab3e1233f73', 'request_id': 'req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:41.755444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd68146e14c8d42cc9650eab3e1233f73', 'request_id': 'req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:41.844655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd68146e14c8d42cc9650eab3e1233f73', 'request_id': 'req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:41.856641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] The token cache is being invalidate because project d68146e14c8d42cc9650eab3e1233f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:36:41.856925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:41.857577 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:41.859132 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1394/2799] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:36:41 2022] DELETE /identity/v3/projects/d68146e14c8d42cc9650eab3e1233f73 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:41.863214 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cefcff-d8c2-4ae4-9cbe-dc476938102c service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:41.873018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cefcff-d8c2-4ae4-9cbe-dc476938102c service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:41.928749 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:41.929628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:41.929946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:41.930232 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] PATH_INFO: `/v3/users/9281156219c64c87926834f5d8df6c3a` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:41.931422 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9281156219c64c87926834f5d8df6c3a)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:41.933424 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:42.105995 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9281156219c64c87926834f5d8df6c3a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.106664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9281156219c64c87926834f5d8df6c3a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.190550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9281156219c64c87926834f5d8df6c3a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.203514 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9281156219c64c87926834f5d8df6c3a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.204265 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-e3a158bb-659d-4654-9ea6-dda1188ec594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9281156219c64c87926834f5d8df6c3a'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.226372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1406/2800] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:36:41 2022] DELETE /identity/v3/users/9281156219c64c87926834f5d8df6c3a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:42.665193 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:42.667908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:42.681112 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6fd6f-5fe9-4991-8180-9938ded44c4d admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:42.758005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:42.760146 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:42.760702 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:42.761181 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] PATH_INFO: `/v3/projects/2754d8ba11834ab4a55373bd0d97ea12` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:42.762622 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2754d8ba11834ab4a55373bd0d97ea12)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:42.764559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:42.881638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2754d8ba11834ab4a55373bd0d97ea12', 'request_id': 'req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.887031 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2754d8ba11834ab4a55373bd0d97ea12', 'request_id': 'req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.982794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2754d8ba11834ab4a55373bd0d97ea12', 'request_id': 'req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.995736 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] The token cache is being invalidate because project 2754d8ba11834ab4a55373bd0d97ea12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80277) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:36:42.996259 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.996997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-4d362dd7-6806-40d3-b5a3-3a2ce0486e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:42.998987 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1395/2801] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:36:42 2022] DELETE /identity/v3/projects/2754d8ba11834ab4a55373bd0d97ea12 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:43.003554 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512664b1-cb14-48c9-8da9-d86129eac4e5 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.004042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512664b1-cb14-48c9-8da9-d86129eac4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.004302 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512664b1-cb14-48c9-8da9-d86129eac4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.022423 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-512664b1-cb14-48c9-8da9-d86129eac4e5 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:43.023917 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-512664b1-cb14-48c9-8da9-d86129eac4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:43.095674 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1407/2802] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:36:43.097299 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854358-2b43-45e6-9334-a70bdb659582 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.097591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854358-2b43-45e6-9334-a70bdb659582 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.097903 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51854358-2b43-45e6-9334-a70bdb659582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.117376 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-51854358-2b43-45e6-9334-a70bdb659582 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:43.118641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-51854358-2b43-45e6-9334-a70bdb659582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:43.133840 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1396/2803] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:36:43.139118 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512664b1-cb14-48c9-8da9-d86129eac4e5 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:43.148794 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512664b1-cb14-48c9-8da9-d86129eac4e5 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:43.154102 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:43.154792 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.155077 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.155483 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] PATH_INFO: `/v3/domains` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.156213 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80278) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:36:43.156519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:43.158144 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:43.170206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1408/2804] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:36:43.174611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51854358-2b43-45e6-9334-a70bdb659582 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:43.183950 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51854358-2b43-45e6-9334-a70bdb659582 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:43.191993 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9709110-559e-450b-9545-57a4c6139bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:43.192802 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9709110-559e-450b-9545-57a4c6139bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.193130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9709110-559e-450b-9545-57a4c6139bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.195647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9709110-559e-450b-9545-57a4c6139bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.196375 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9709110-559e-450b-9545-57a4c6139bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:43.198937 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9709110-559e-450b-9545-57a4c6139bd8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:43.381451 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1397/2805] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:36:43 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:43.382682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:43.393458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735fd32-e007-4afa-9181-dec6301c113e admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:43.447859 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8bd7b-ada3-4efe-ba4e-7c4b9489c51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:43.449256 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8bd7b-ada3-4efe-ba4e-7c4b9489c51a admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.449481 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8bd7b-ada3-4efe-ba4e-7c4b9489c51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.449750 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8bd7b-ada3-4efe-ba4e-7c4b9489c51a admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.450536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8bd7b-ada3-4efe-ba4e-7c4b9489c51a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:43.452505 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8bd7b-ada3-4efe-ba4e-7c4b9489c51a admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:43.530817 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1409/2806] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:36:43 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:36:43.532104 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9709110-559e-450b-9545-57a4c6139bd8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:43.542713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9709110-559e-450b-9545-57a4c6139bd8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:43.547997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffec17-d499-49ab-afc9-8a9b39d26d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:43.548264 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffec17-d499-49ab-afc9-8a9b39d26d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.548557 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffec17-d499-49ab-afc9-8a9b39d26d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.548871 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffec17-d499-49ab-afc9-8a9b39d26d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.549617 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffec17-d499-49ab-afc9-8a9b39d26d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:43.551353 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffec17-d499-49ab-afc9-8a9b39d26d1e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:43.563914 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1398/2807] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:43.568858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8bd7b-ada3-4efe-ba4e-7c4b9489c51a admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:43.579671 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8bd7b-ada3-4efe-ba4e-7c4b9489c51a admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:43.584326 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d97eaa-ed1e-4a7e-ba75-d561d5374642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:43.585061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d97eaa-ed1e-4a7e-ba75-d561d5374642 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.585414 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d97eaa-ed1e-4a7e-ba75-d561d5374642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.585758 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d97eaa-ed1e-4a7e-ba75-d561d5374642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.586783 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d97eaa-ed1e-4a7e-ba75-d561d5374642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:43.588412 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d97eaa-ed1e-4a7e-ba75-d561d5374642 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:43.601881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1410/2808] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:43.603087 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffec17-d499-49ab-afc9-8a9b39d26d1e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:43.613399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffec17-d499-49ab-afc9-8a9b39d26d1e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:43.618321 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40231884-302e-47e7-931f-e1445d9460da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:43.619205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40231884-302e-47e7-931f-e1445d9460da admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.619487 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40231884-302e-47e7-931f-e1445d9460da admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.619784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40231884-302e-47e7-931f-e1445d9460da admin admin] PATH_INFO: `/v3/projects/573a49f943d2476e8202287dc856488a/users/4dfd53a6f6c943df8a61d126d75b5802/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.633923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40231884-302e-47e7-931f-e1445d9460da admin admin] RBAC: Authorizing `identity:create_grant(project_id=573a49f943d2476e8202287dc856488a, user_id=4dfd53a6f6c943df8a61d126d75b5802, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:43.636595 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40231884-302e-47e7-931f-e1445d9460da admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:43.736429 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1399/2809] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:43 2022] PUT /identity/v3/projects/573a49f943d2476e8202287dc856488a/users/4dfd53a6f6c943df8a61d126d75b5802/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:43.748153 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd562f6-827d-4549-806b-0ed31e901d08 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.748451 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd562f6-827d-4549-806b-0ed31e901d08 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.748739 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd562f6-827d-4549-806b-0ed31e901d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.775228 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-8bd562f6-827d-4549-806b-0ed31e901d08 None None] MFA Rules not processed for user `4dfd53a6f6c943df8a61d126d75b5802`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:43.777098 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-8bd562f6-827d-4549-806b-0ed31e901d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:43.839225 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1411/2810] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:43.843804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40231884-302e-47e7-931f-e1445d9460da admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:43.907013 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40231884-302e-47e7-931f-e1445d9460da admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:43.911710 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0560f71-ccd4-4a8b-871f-4106aa33244b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:43.912374 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560f71-ccd4-4a8b-871f-4106aa33244b admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:43.912660 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560f71-ccd4-4a8b-871f-4106aa33244b admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:43.912942 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560f71-ccd4-4a8b-871f-4106aa33244b admin admin] PATH_INFO: `/v3/projects` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:43.913647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0560f71-ccd4-4a8b-871f-4106aa33244b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:43.915603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0560f71-ccd4-4a8b-871f-4106aa33244b admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:44.038017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1400/2811] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:36:43 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:44.039173 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd562f6-827d-4549-806b-0ed31e901d08 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:44.050266 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd562f6-827d-4549-806b-0ed31e901d08 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:44.105737 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b3371-beae-4459-a40f-15dd2053a1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:44.106439 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b3371-beae-4459-a40f-15dd2053a1ea admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:44.106724 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b3371-beae-4459-a40f-15dd2053a1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:44.107002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b3371-beae-4459-a40f-15dd2053a1ea admin admin] PATH_INFO: `/v3/users` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:44.107683 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b3371-beae-4459-a40f-15dd2053a1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:44.109600 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b3371-beae-4459-a40f-15dd2053a1ea admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:44.214251 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1412/2812] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:36:44 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:36:44.218323 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0560f71-ccd4-4a8b-871f-4106aa33244b admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:44.228441 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0560f71-ccd4-4a8b-871f-4106aa33244b admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:44.233747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d7a4c-96ce-4482-8b14-9ba7c932e601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:44.234460 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d7a4c-96ce-4482-8b14-9ba7c932e601 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:44.234742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d7a4c-96ce-4482-8b14-9ba7c932e601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:44.235018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d7a4c-96ce-4482-8b14-9ba7c932e601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:44.235742 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d7a4c-96ce-4482-8b14-9ba7c932e601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:44.237336 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d7a4c-96ce-4482-8b14-9ba7c932e601 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:44.248184 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1401/2813] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:44.252019 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b3371-beae-4459-a40f-15dd2053a1ea admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:44.262652 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b3371-beae-4459-a40f-15dd2053a1ea admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:44.267079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906285a-e7ae-4085-b7d9-2c72cfda6d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:44.267972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906285a-e7ae-4085-b7d9-2c72cfda6d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:44.268257 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906285a-e7ae-4085-b7d9-2c72cfda6d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:44.268536 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906285a-e7ae-4085-b7d9-2c72cfda6d65 admin admin] PATH_INFO: `/v3/projects/dc9de40a6ddf492b8c1c93be667cfe2f/users/7911f95fde0c4ac897f597a92bf3a385/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:44.283058 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906285a-e7ae-4085-b7d9-2c72cfda6d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9de40a6ddf492b8c1c93be667cfe2f, user_id=7911f95fde0c4ac897f597a92bf3a385, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:44.285753 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906285a-e7ae-4085-b7d9-2c72cfda6d65 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:44.360495 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1413/2814] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:44 2022] PUT /identity/v3/projects/dc9de40a6ddf492b8c1c93be667cfe2f/users/7911f95fde0c4ac897f597a92bf3a385/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:36:44.370483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d7a4c-96ce-4482-8b14-9ba7c932e601 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:44.380835 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d7a4c-96ce-4482-8b14-9ba7c932e601 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:44.450867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8a4bd-974c-4bb4-9b4c-e0038aa4e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:44.451605 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a4bd-974c-4bb4-9b4c-e0038aa4e8df admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:44.451883 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a4bd-974c-4bb4-9b4c-e0038aa4e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:44.452154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a4bd-974c-4bb4-9b4c-e0038aa4e8df admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:44.452881 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8a4bd-974c-4bb4-9b4c-e0038aa4e8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:44.454536 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8a4bd-974c-4bb4-9b4c-e0038aa4e8df admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:44.465654 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1402/2815] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:36:44.469936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906285a-e7ae-4085-b7d9-2c72cfda6d65 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:44.481718 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906285a-e7ae-4085-b7d9-2c72cfda6d65 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:44.488048 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:44.489067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:44.489421 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:44.489770 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] PATH_INFO: `/v3/projects/dc9de40a6ddf492b8c1c93be667cfe2f/users/7911f95fde0c4ac897f597a92bf3a385/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:44.491885 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9de40a6ddf492b8c1c93be667cfe2f, user_id=7911f95fde0c4ac897f597a92bf3a385, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:44.495776 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:44.507866 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1414/2816] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:36:44 2022] PUT /identity/v3/projects/dc9de40a6ddf492b8c1c93be667cfe2f/users/7911f95fde0c4ac897f597a92bf3a385/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:36:44.518244 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584f99a-0831-4950-8ca3-41d851d3c61b None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:44.518556 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584f99a-0831-4950-8ca3-41d851d3c61b None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:44.518931 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584f99a-0831-4950-8ca3-41d851d3c61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:44.547051 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-f584f99a-0831-4950-8ca3-41d851d3c61b None None] MFA Rules not processed for user `7911f95fde0c4ac897f597a92bf3a385`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:36:44.551022 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f584f99a-0831-4950-8ca3-41d851d3c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:44.661205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1403/2817] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:36:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:36:44.675334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:44.677647 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:44.737067 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1884cd1-d51f-4897-964a-daa91ca1ede5 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:44.741113 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68979a0b-832d-4754-91f9-db64d642ed07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:44.742322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68979a0b-832d-4754-91f9-db64d642ed07 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:44.742613 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68979a0b-832d-4754-91f9-db64d642ed07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:44.742896 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68979a0b-832d-4754-91f9-db64d642ed07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:44.799978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68979a0b-832d-4754-91f9-db64d642ed07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:44.801357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68979a0b-832d-4754-91f9-db64d642ed07 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:44.810992 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1415/2818] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:47.367715 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584f99a-0831-4950-8ca3-41d851d3c61b None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:47.369345 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-f584f99a-0831-4950-8ca3-41d851d3c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:47.434005 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584f99a-0831-4950-8ca3-41d851d3c61b None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:47.438746 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:47.439636 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:47.439921 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:47.440198 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] PATH_INFO: `/v3/users/8101850dd18f4108ac12ff9487cbbe3e` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:47.441398 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8101850dd18f4108ac12ff9487cbbe3e)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:47.443505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:47.637402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8101850dd18f4108ac12ff9487cbbe3e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:47.638145 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8101850dd18f4108ac12ff9487cbbe3e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:47.638766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8101850dd18f4108ac12ff9487cbbe3e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:47.645602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8101850dd18f4108ac12ff9487cbbe3e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:47.710166 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8101850dd18f4108ac12ff9487cbbe3e'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:47.727283 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1404/2819] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:36:47 2022] DELETE /identity/v3/users/8101850dd18f4108ac12ff9487cbbe3e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:48.008880 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68979a0b-832d-4754-91f9-db64d642ed07 service cinder] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:48.019961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68979a0b-832d-4754-91f9-db64d642ed07 service cinder] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:48.075828 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:48.076689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:48.076977 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:48.077254 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] PATH_INFO: `/v3/projects/d3e32e04d1f44e97b903f98c7c2749db` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:48.078455 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e32e04d1f44e97b903f98c7c2749db)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:48.080372 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:48.192849 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e32e04d1f44e97b903f98c7c2749db', 'request_id': 'req-a4abd200-34c1-4c8a-bc11-64978d162ca3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.198147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e32e04d1f44e97b903f98c7c2749db', 'request_id': 'req-a4abd200-34c1-4c8a-bc11-64978d162ca3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.275861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e32e04d1f44e97b903f98c7c2749db', 'request_id': 'req-a4abd200-34c1-4c8a-bc11-64978d162ca3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.287358 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] The token cache is being invalidate because project d3e32e04d1f44e97b903f98c7c2749db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:36:48.288891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.288891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.289743 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1416/2820] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:36:48 2022] DELETE /identity/v3/projects/d3e32e04d1f44e97b903f98c7c2749db => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:48.294289 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:48.295690 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:48.308984 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dafaf-70ad-4c5d-b9b7-68e39f086fbb admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:48.363046 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:48.363923 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:48.364207 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:48.364484 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] PATH_INFO: `/v3/users/13cac6c04c624feab53852403f56b011` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:48.365674 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13cac6c04c624feab53852403f56b011)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:48.367669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:48.544599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13cac6c04c624feab53852403f56b011'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.544599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13cac6c04c624feab53852403f56b011'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.544599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13cac6c04c624feab53852403f56b011'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.549664 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13cac6c04c624feab53852403f56b011'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.613280 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13cac6c04c624feab53852403f56b011'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:48.630813 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1405/2821] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:36:48 2022] DELETE /identity/v3/users/13cac6c04c624feab53852403f56b011 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:36:48.882843 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:48.884281 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:48.903081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd200-34c1-4c8a-bc11-64978d162ca3 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:48.957361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:48.958633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:48.958922 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:48.959202 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] PATH_INFO: `/v3/projects/9cbe2e9981c749b29a3a0d9b2d6d4db3` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:48.960417 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cbe2e9981c749b29a3a0d9b2d6d4db3)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:48.962626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:49.093723 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cbe2e9981c749b29a3a0d9b2d6d4db3', 'request_id': 'req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:49.099976 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cbe2e9981c749b29a3a0d9b2d6d4db3', 'request_id': 'req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:49.167470 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cbe2e9981c749b29a3a0d9b2d6d4db3', 'request_id': 'req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:49.180051 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] The token cache is being invalidate because project 9cbe2e9981c749b29a3a0d9b2d6d4db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:36:49.180334 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:49.180965 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:36:49.182550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1417/2822] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:36:48 2022] DELETE /identity/v3/projects/9cbe2e9981c749b29a3a0d9b2d6d4db3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:36:51.237291 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:51.238719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:51.293433 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aaae1-8738-402c-80a5-62f3bfc4f222 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:51.338343 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:51.338981 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:51.339263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:51.339603 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:51.341870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:51.401411 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:51.403435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:51.412902 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1406/2823] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:53.677039 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:53.678717 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:53.805398 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c272498-e4d7-4df2-8c77-941b3f1c1dd3 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:53.811222 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:53.811933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:53.812220 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:53.812522 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:53.814332 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:53.826531 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:53.828120 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:53.839972 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1418/2824] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:36:56.446841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:36:56.493103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a31f9-bc80-4440-a5f0-219192952e59 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:36:56.499714 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f94e52e32324d8cbf38fb05dc38ea96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:36:56.500387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:36:56.500730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:36:56.501018 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:36:56.502938 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:36:56.574134 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:36:56.576347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:36:56.585881 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1407/2825] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:00.390357 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:00.391872 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:00.469534 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ee4eb-2499-47b9-8848-ad7c95c4a1e2 service nova] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:00.473259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:00.473942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:00.474231 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:00.474562 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:00.476322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:00.560367 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:00.561857 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:00.580513 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1419/2826] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:07.980521 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:07.982040 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:07.996599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754b136-161e-4080-8b0c-b91790377065 service nova] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:08.001288 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:08.002186 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:08.002648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:08.002935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] PATH_INFO: `/v3/users/9c6a8bd3014349a3aa60c9471f3c8234` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:08.004154 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c6a8bd3014349a3aa60c9471f3c8234)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:08.006226 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:08.218085 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c6a8bd3014349a3aa60c9471f3c8234'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.218667 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c6a8bd3014349a3aa60c9471f3c8234'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.219263 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c6a8bd3014349a3aa60c9471f3c8234'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.226148 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c6a8bd3014349a3aa60c9471f3c8234'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.301961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c6a8bd3014349a3aa60c9471f3c8234'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.319399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1408/2827] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:37:07 2022] DELETE /identity/v3/users/9c6a8bd3014349a3aa60c9471f3c8234 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:08.578285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:08.591655 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e75e6-ea1a-42d6-bde3-617db80a53a4 service neutron] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:08.650865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:08.651779 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:08.652068 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:08.652346 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] PATH_INFO: `/v3/projects/0b1cc4c699be4edfa243bc7dde91cd51` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:08.653543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1cc4c699be4edfa243bc7dde91cd51)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:08.655540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:08.754948 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1cc4c699be4edfa243bc7dde91cd51', 'request_id': 'req-c7cda9dd-40f8-4925-adf9-626770ee4955'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.760059 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1cc4c699be4edfa243bc7dde91cd51', 'request_id': 'req-c7cda9dd-40f8-4925-adf9-626770ee4955'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.895811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1cc4c699be4edfa243bc7dde91cd51', 'request_id': 'req-c7cda9dd-40f8-4925-adf9-626770ee4955'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.907608 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] The token cache is being invalidate because project 0b1cc4c699be4edfa243bc7dde91cd51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:37:08.907897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.908523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:08.910158 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1420/2828] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:37:08 2022] DELETE /identity/v3/projects/0b1cc4c699be4edfa243bc7dde91cd51 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:08.914311 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:08.915750 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:08.929045 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991b10c-87ae-4588-9471-58c9ff7ae879 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:08.984340 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:08.985205 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:08.985493 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:08.985773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] PATH_INFO: `/v3/users/f72e83f97c884e4b93df99535e66a3cc` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:08.986997 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72e83f97c884e4b93df99535e66a3cc)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:08.989229 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:09.155094 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72e83f97c884e4b93df99535e66a3cc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.155705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72e83f97c884e4b93df99535e66a3cc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.156310 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72e83f97c884e4b93df99535e66a3cc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.162875 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72e83f97c884e4b93df99535e66a3cc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.234890 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72e83f97c884e4b93df99535e66a3cc'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.251915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1409/2829] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:37:08 2022] DELETE /identity/v3/users/f72e83f97c884e4b93df99535e66a3cc => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:09.528858 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:09.530321 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:09.542755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cda9dd-40f8-4925-adf9-626770ee4955 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:09.604156 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:09.606081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:09.606081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:09.606081 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] PATH_INFO: `/v3/projects/81a2c144876d4dfc8b4d4ed76426fdbf` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:09.606980 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a2c144876d4dfc8b4d4ed76426fdbf)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:09.608881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:09.766716 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a2c144876d4dfc8b4d4ed76426fdbf', 'request_id': 'req-f473f2ef-335d-40a3-9118-8710a9f97dee'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.771928 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a2c144876d4dfc8b4d4ed76426fdbf', 'request_id': 'req-f473f2ef-335d-40a3-9118-8710a9f97dee'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.850097 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a2c144876d4dfc8b4d4ed76426fdbf', 'request_id': 'req-f473f2ef-335d-40a3-9118-8710a9f97dee'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.861186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] The token cache is being invalidate because project 81a2c144876d4dfc8b4d4ed76426fdbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:37:09.861467 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.862091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-f473f2ef-335d-40a3-9118-8710a9f97dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:09.863606 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1421/2830] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:37:09 2022] DELETE /identity/v3/projects/81a2c144876d4dfc8b4d4ed76426fdbf => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:18.465930 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:18.467290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:18.480391 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0204665-f08d-4502-b197-c3b6226fae14 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:18.534925 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:18.535788 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:18.536067 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:18.536367 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] PATH_INFO: `/v3/users/4dfd53a6f6c943df8a61d126d75b5802` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:18.537544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dfd53a6f6c943df8a61d126d75b5802)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:18.539489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:18.720469 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dfd53a6f6c943df8a61d126d75b5802'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:18.721129 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dfd53a6f6c943df8a61d126d75b5802'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:18.721734 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dfd53a6f6c943df8a61d126d75b5802'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:18.728480 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dfd53a6f6c943df8a61d126d75b5802'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:18.805773 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dfd53a6f6c943df8a61d126d75b5802'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:18.817882 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1410/2831] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:37:18 2022] DELETE /identity/v3/users/4dfd53a6f6c943df8a61d126d75b5802 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:18.822479 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b0bec-20d1-44b0-9780-f5d9acbad4a8 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:18.822946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b0bec-20d1-44b0-9780-f5d9acbad4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:18.823060 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b0bec-20d1-44b0-9780-f5d9acbad4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:18.849198 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-2a9b0bec-20d1-44b0-9780-f5d9acbad4a8 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:18.850444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2a9b0bec-20d1-44b0-9780-f5d9acbad4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:18.907150 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1422/2832] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:18.916781 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:18.918847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:18.975602 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55e236-3697-4aa2-b5b1-a6921aab82e1 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:18.979161 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609f540-9d01-4ce5-b248-ea8123c4956d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:18.980348 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609f540-9d01-4ce5-b248-ea8123c4956d service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:18.980630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609f540-9d01-4ce5-b248-ea8123c4956d service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:18.980909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609f540-9d01-4ce5-b248-ea8123c4956d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:18.989687 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609f540-9d01-4ce5-b248-ea8123c4956d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:18.991196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609f540-9d01-4ce5-b248-ea8123c4956d service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:19.001196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1411/2833] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:19.417713 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b0bec-20d1-44b0-9780-f5d9acbad4a8 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:19.419344 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-2a9b0bec-20d1-44b0-9780-f5d9acbad4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:19.433026 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b0bec-20d1-44b0-9780-f5d9acbad4a8 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:19.439459 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:19.440335 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:19.440628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:19.440978 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] PATH_INFO: `/v3/projects/573a49f943d2476e8202287dc856488a` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:19.442196 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=573a49f943d2476e8202287dc856488a)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:19.444143 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:19.586616 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573a49f943d2476e8202287dc856488a', 'request_id': 'req-feddcbf4-e86d-43db-9f6a-c155e98267f7'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.592625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573a49f943d2476e8202287dc856488a', 'request_id': 'req-feddcbf4-e86d-43db-9f6a-c155e98267f7'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.669279 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573a49f943d2476e8202287dc856488a', 'request_id': 'req-feddcbf4-e86d-43db-9f6a-c155e98267f7'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.680092 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] The token cache is being invalidate because project 573a49f943d2476e8202287dc856488a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:37:19.680385 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.681004 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.682632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1423/2834] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:37:19 2022] DELETE /identity/v3/projects/573a49f943d2476e8202287dc856488a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:19.686747 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609f540-9d01-4ce5-b248-ea8123c4956d service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:19.696334 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609f540-9d01-4ce5-b248-ea8123c4956d service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:19.751079 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:19.751866 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:19.752153 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:19.752432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] PATH_INFO: `/v3/users/7911f95fde0c4ac897f597a92bf3a385` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:19.753620 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7911f95fde0c4ac897f597a92bf3a385)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:19.755582 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:19.891584 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7911f95fde0c4ac897f597a92bf3a385'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.892237 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7911f95fde0c4ac897f597a92bf3a385'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.892862 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7911f95fde0c4ac897f597a92bf3a385'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.899423 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7911f95fde0c4ac897f597a92bf3a385'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.972402 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-fd5fb233-ddf4-48c5-baa9-c5e4b2a62ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7911f95fde0c4ac897f597a92bf3a385'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:19.989489 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1412/2835] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:37:19 2022] DELETE /identity/v3/users/7911f95fde0c4ac897f597a92bf3a385 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:20.389347 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:20.390853 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:20.403594 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddcbf4-e86d-43db-9f6a-c155e98267f7 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:20.467094 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:20.467970 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:20.468294 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:20.468571 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] PATH_INFO: `/v3/projects/dc9de40a6ddf492b8c1c93be667cfe2f` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:20.469897 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9de40a6ddf492b8c1c93be667cfe2f)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:20.471910 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:20.577285 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9de40a6ddf492b8c1c93be667cfe2f', 'request_id': 'req-69620b9f-1755-4f1c-b77a-203a40d48ec4'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:20.582420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9de40a6ddf492b8c1c93be667cfe2f', 'request_id': 'req-69620b9f-1755-4f1c-b77a-203a40d48ec4'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:20.628998 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9de40a6ddf492b8c1c93be667cfe2f', 'request_id': 'req-69620b9f-1755-4f1c-b77a-203a40d48ec4'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:20.640389 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] The token cache is being invalidate because project dc9de40a6ddf492b8c1c93be667cfe2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:37:20.640621 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:20.641249 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-69620b9f-1755-4f1c-b77a-203a40d48ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:20.642812 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1424/2836] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:37:20 2022] DELETE /identity/v3/projects/dc9de40a6ddf492b8c1c93be667cfe2f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:20.647649 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab9346-72ec-41ab-adff-604fc6352a0c None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:20.647940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab9346-72ec-41ab-adff-604fc6352a0c None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:20.648224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab9346-72ec-41ab-adff-604fc6352a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:20.694057 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-76ab9346-72ec-41ab-adff-604fc6352a0c None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:20.695247 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-76ab9346-72ec-41ab-adff-604fc6352a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:20.752213 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1413/2837] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:20.760093 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bcd63-6bf6-43ca-a95b-1b663901cfc3 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:20.760392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bcd63-6bf6-43ca-a95b-1b663901cfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:20.760679 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bcd63-6bf6-43ca-a95b-1b663901cfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:20.779550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-a51bcd63-6bf6-43ca-a95b-1b663901cfc3 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:20.780771 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-a51bcd63-6bf6-43ca-a95b-1b663901cfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:20.788689 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1425/2838] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:20.792841 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab9346-72ec-41ab-adff-604fc6352a0c None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:20.802415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab9346-72ec-41ab-adff-604fc6352a0c None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:20.807017 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:20.808224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:20.808642 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:20.808928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:20.809657 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:37:20.809996 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:20.811689 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:20.823262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1414/2839] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:37:20.829933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bcd63-6bf6-43ca-a95b-1b663901cfc3 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:20.840013 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bcd63-6bf6-43ca-a95b-1b663901cfc3 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:20.844968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c76a3-1a6d-42db-8378-cca9422a14a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:20.845628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c76a3-1a6d-42db-8378-cca9422a14a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:20.846014 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c76a3-1a6d-42db-8378-cca9422a14a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:20.846299 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c76a3-1a6d-42db-8378-cca9422a14a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:20.850782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c76a3-1a6d-42db-8378-cca9422a14a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:20.850782 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c76a3-1a6d-42db-8378-cca9422a14a0 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:20.959252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1426/2840] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:37:20 2022] POST /identity/v3/projects => generated 401 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:37:20.961196 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:20.971415 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fb757-bd92-494e-90e6-4f415964e384 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:21.026683 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec83db-4d83-4ab1-a827-76b0f66ee249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:21.027347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec83db-4d83-4ab1-a827-76b0f66ee249 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.027635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec83db-4d83-4ab1-a827-76b0f66ee249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.027918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec83db-4d83-4ab1-a827-76b0f66ee249 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.028611 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec83db-4d83-4ab1-a827-76b0f66ee249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:21.030574 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec83db-4d83-4ab1-a827-76b0f66ee249 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:21.109349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1415/2841] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:37:20 2022] POST /identity/v3/users => generated 301 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:37:21.113442 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c76a3-1a6d-42db-8378-cca9422a14a0 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:21.123971 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c76a3-1a6d-42db-8378-cca9422a14a0 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:21.129523 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba83fa-1e14-4390-b8ef-9a09a66aee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:21.130207 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba83fa-1e14-4390-b8ef-9a09a66aee80 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.130494 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba83fa-1e14-4390-b8ef-9a09a66aee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.130772 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba83fa-1e14-4390-b8ef-9a09a66aee80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.131498 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba83fa-1e14-4390-b8ef-9a09a66aee80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:21.133147 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba83fa-1e14-4390-b8ef-9a09a66aee80 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:21.143690 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1427/2842] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:37:21.147585 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec83db-4d83-4ab1-a827-76b0f66ee249 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:21.157628 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec83db-4d83-4ab1-a827-76b0f66ee249 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:21.162214 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae9333-e399-456b-82d8-727919bd18e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:21.162861 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae9333-e399-456b-82d8-727919bd18e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.163139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae9333-e399-456b-82d8-727919bd18e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.163413 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae9333-e399-456b-82d8-727919bd18e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.164135 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae9333-e399-456b-82d8-727919bd18e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:21.165733 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae9333-e399-456b-82d8-727919bd18e4 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:21.175850 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1416/2843] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 13 13:37:21.179675 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba83fa-1e14-4390-b8ef-9a09a66aee80 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:21.189920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba83fa-1e14-4390-b8ef-9a09a66aee80 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:21.194540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50e587-c542-44ae-ad52-aef08dc18c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:21.195409 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50e587-c542-44ae-ad52-aef08dc18c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.195694 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50e587-c542-44ae-ad52-aef08dc18c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.195974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50e587-c542-44ae-ad52-aef08dc18c37 admin admin] PATH_INFO: `/v3/projects/2827d5609bdd47dcace9c2fa9b01e122/users/c52e14ab3f554352993c999981861209/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.210206 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50e587-c542-44ae-ad52-aef08dc18c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2827d5609bdd47dcace9c2fa9b01e122, user_id=c52e14ab3f554352993c999981861209, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:21.213032 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50e587-c542-44ae-ad52-aef08dc18c37 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:21.280975 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1428/2844] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:37:21 2022] PUT /identity/v3/projects/2827d5609bdd47dcace9c2fa9b01e122/users/c52e14ab3f554352993c999981861209/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:21.340575 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aba5d7-1a7f-43d2-a52b-3e0afdada4ac None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.340983 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aba5d7-1a7f-43d2-a52b-3e0afdada4ac None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.341271 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aba5d7-1a7f-43d2-a52b-3e0afdada4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.367473 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-a4aba5d7-1a7f-43d2-a52b-3e0afdada4ac None None] MFA Rules not processed for user `c52e14ab3f554352993c999981861209`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:21.368749 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-a4aba5d7-1a7f-43d2-a52b-3e0afdada4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:21.430273 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1417/2845] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:21.435371 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50e587-c542-44ae-ad52-aef08dc18c37 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:21.494633 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50e587-c542-44ae-ad52-aef08dc18c37 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:21.499966 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73771e-5ad5-456f-a089-a824b1587cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:21.500625 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73771e-5ad5-456f-a089-a824b1587cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.500915 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73771e-5ad5-456f-a089-a824b1587cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.501197 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73771e-5ad5-456f-a089-a824b1587cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.501937 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73771e-5ad5-456f-a089-a824b1587cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:21.503895 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73771e-5ad5-456f-a089-a824b1587cfb admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:21.599401 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1429/2846] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:37:21 2022] POST /identity/v3/projects => generated 401 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:37:21.603485 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aba5d7-1a7f-43d2-a52b-3e0afdada4ac None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:21.614771 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aba5d7-1a7f-43d2-a52b-3e0afdada4ac None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:21.668724 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b41b1-43a9-4660-b07b-a745eeb470a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:21.669380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b41b1-43a9-4660-b07b-a745eeb470a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.669669 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b41b1-43a9-4660-b07b-a745eeb470a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.669978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b41b1-43a9-4660-b07b-a745eeb470a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.670666 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b41b1-43a9-4660-b07b-a745eeb470a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:21.672573 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b41b1-43a9-4660-b07b-a745eeb470a2 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:21.776936 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1418/2847] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:37:21 2022] POST /identity/v3/users => generated 307 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:37:21.780822 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73771e-5ad5-456f-a089-a824b1587cfb admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:21.791002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73771e-5ad5-456f-a089-a824b1587cfb admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:21.796359 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e33c09-47a4-41ea-91b2-b66729bfc535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:21.797069 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e33c09-47a4-41ea-91b2-b66729bfc535 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.797324 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e33c09-47a4-41ea-91b2-b66729bfc535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.797602 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e33c09-47a4-41ea-91b2-b66729bfc535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.798368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e33c09-47a4-41ea-91b2-b66729bfc535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:21.800297 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e33c09-47a4-41ea-91b2-b66729bfc535 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:21.813191 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1430/2848] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:37:21.817591 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b41b1-43a9-4660-b07b-a745eeb470a2 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:21.828210 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b41b1-43a9-4660-b07b-a745eeb470a2 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:21.832820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc68aa-f11f-4e2e-8034-ac8e3b50a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:21.833694 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc68aa-f11f-4e2e-8034-ac8e3b50a08c admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:21.834008 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc68aa-f11f-4e2e-8034-ac8e3b50a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:21.834285 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc68aa-f11f-4e2e-8034-ac8e3b50a08c admin admin] PATH_INFO: `/v3/projects/3caf8d8c8d1448379db98762afd4d155/users/e328e4d952e74280abc9d1176a4aaf03/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:21.848559 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc68aa-f11f-4e2e-8034-ac8e3b50a08c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3caf8d8c8d1448379db98762afd4d155, user_id=e328e4d952e74280abc9d1176a4aaf03, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:21.852121 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc68aa-f11f-4e2e-8034-ac8e3b50a08c admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:21.959223 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1419/2849] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:37:21 2022] PUT /identity/v3/projects/3caf8d8c8d1448379db98762afd4d155/users/e328e4d952e74280abc9d1176a4aaf03/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:37:21.963406 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e33c09-47a4-41ea-91b2-b66729bfc535 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:21.973246 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e33c09-47a4-41ea-91b2-b66729bfc535 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:22.026692 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162beb20-293d-4f40-bfb5-7368ef34fc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:22.027411 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162beb20-293d-4f40-bfb5-7368ef34fc74 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:22.027643 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162beb20-293d-4f40-bfb5-7368ef34fc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:22.027920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162beb20-293d-4f40-bfb5-7368ef34fc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:22.028639 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162beb20-293d-4f40-bfb5-7368ef34fc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:22.030282 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162beb20-293d-4f40-bfb5-7368ef34fc74 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:22.039605 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1431/2850] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:37:22.044206 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc68aa-f11f-4e2e-8034-ac8e3b50a08c admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:22.054309 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc68aa-f11f-4e2e-8034-ac8e3b50a08c admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:22.059422 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:22.060290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:22.060570 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:22.060845 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] PATH_INFO: `/v3/projects/3caf8d8c8d1448379db98762afd4d155/users/e328e4d952e74280abc9d1176a4aaf03/roles/663341777b0b44128ed78fe8d40f6462` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:22.062819 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3caf8d8c8d1448379db98762afd4d155, user_id=e328e4d952e74280abc9d1176a4aaf03, role_id=663341777b0b44128ed78fe8d40f6462)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:22.065444 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:22.075165 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1420/2851] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:37:22 2022] PUT /identity/v3/projects/3caf8d8c8d1448379db98762afd4d155/users/e328e4d952e74280abc9d1176a4aaf03/roles/663341777b0b44128ed78fe8d40f6462 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:22.085059 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8552bd-dac9-4524-b72c-ec321b8ed688 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:22.085353 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8552bd-dac9-4524-b72c-ec321b8ed688 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:22.085634 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8552bd-dac9-4524-b72c-ec321b8ed688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:22.113676 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ff8552bd-dac9-4524-b72c-ec321b8ed688 None None] MFA Rules not processed for user `e328e4d952e74280abc9d1176a4aaf03`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:22.114961 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ff8552bd-dac9-4524-b72c-ec321b8ed688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:22.180506 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1432/2852] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:22.191688 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:22.193404 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:22.248541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6016e-59fb-413e-b7aa-6128f5980ce8 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:22.252152 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d7839-b042-44d6-8885-c2a12a058c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:22.252783 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7839-b042-44d6-8885-c2a12a058c64 service cinder] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:22.253071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7839-b042-44d6-8885-c2a12a058c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:22.253349 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7839-b042-44d6-8885-c2a12a058c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:22.263626 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d7839-b042-44d6-8885-c2a12a058c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:22.265048 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d7839-b042-44d6-8885-c2a12a058c64 service cinder] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:22.274160 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1421/2853] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:22 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:22.473395 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8552bd-dac9-4524-b72c-ec321b8ed688 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:22.483215 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8552bd-dac9-4524-b72c-ec321b8ed688 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:22.486863 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08e5e2-1971-43c6-8945-3aa7677a7a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:22.487503 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e5e2-1971-43c6-8945-3aa7677a7a6e service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:22.487788 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e5e2-1971-43c6-8945-3aa7677a7a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:22.488066 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e5e2-1971-43c6-8945-3aa7677a7a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:22.543024 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08e5e2-1971-43c6-8945-3aa7677a7a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:22.544477 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08e5e2-1971-43c6-8945-3aa7677a7a6e service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:22.553170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1433/2854] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:27.541233 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d7839-b042-44d6-8885-c2a12a058c64 service cinder] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:27.600870 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d7839-b042-44d6-8885-c2a12a058c64 service cinder] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:27.605378 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:27.606262 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:27.606545 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:27.606824 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] PATH_INFO: `/v3/users/c52e14ab3f554352993c999981861209` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:27.607998 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c52e14ab3f554352993c999981861209)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:27.609940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:27.784811 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c52e14ab3f554352993c999981861209'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:27.786589 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c52e14ab3f554352993c999981861209'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:27.787252 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c52e14ab3f554352993c999981861209'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:27.794130 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c52e14ab3f554352993c999981861209'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:27.898179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c52e14ab3f554352993c999981861209'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:27.915401 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1422/2855] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:37:27 2022] DELETE /identity/v3/users/c52e14ab3f554352993c999981861209 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:37:27.920441 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a5b5a-df0c-483c-8df0-2fa38a47a8ef None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:27.920974 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a5b5a-df0c-483c-8df0-2fa38a47a8ef None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:27.921091 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a5b5a-df0c-483c-8df0-2fa38a47a8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:27.938023 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-c57a5b5a-df0c-483c-8df0-2fa38a47a8ef None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:27.939561 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c57a5b5a-df0c-483c-8df0-2fa38a47a8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:27.993587 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1434/2856] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:28.003765 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:28.005169 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:28.059456 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de139a-6aa4-4f5c-9838-2ae07bd95b73 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:28.062990 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e52515-6e7c-40ca-9ce9-70a7b2f590ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:28.063625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e52515-6e7c-40ca-9ce9-70a7b2f590ac service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:28.063906 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e52515-6e7c-40ca-9ce9-70a7b2f590ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:28.064179 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e52515-6e7c-40ca-9ce9-70a7b2f590ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:28.072893 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e52515-6e7c-40ca-9ce9-70a7b2f590ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:28.074395 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e52515-6e7c-40ca-9ce9-70a7b2f590ac service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:28.083713 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1423/2857] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:28.567942 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a5b5a-df0c-483c-8df0-2fa38a47a8ef None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:28.569361 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-c57a5b5a-df0c-483c-8df0-2fa38a47a8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:28.581830 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a5b5a-df0c-483c-8df0-2fa38a47a8ef None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:28.586545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:28.587686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:28.587968 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:28.588245 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] PATH_INFO: `/v3/projects/2827d5609bdd47dcace9c2fa9b01e122` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:28.590551 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2827d5609bdd47dcace9c2fa9b01e122)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:28.591946 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:28.700949 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2827d5609bdd47dcace9c2fa9b01e122', 'request_id': 'req-de96adcd-358a-4a6f-bc53-b80d6fbb3767'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:28.706392 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2827d5609bdd47dcace9c2fa9b01e122', 'request_id': 'req-de96adcd-358a-4a6f-bc53-b80d6fbb3767'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:28.781185 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2827d5609bdd47dcace9c2fa9b01e122', 'request_id': 'req-de96adcd-358a-4a6f-bc53-b80d6fbb3767'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:28.793075 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] The token cache is being invalidate because project 2827d5609bdd47dcace9c2fa9b01e122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:37:28.793362 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:28.794020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:28.795550 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1435/2858] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:37:28 2022] DELETE /identity/v3/projects/2827d5609bdd47dcace9c2fa9b01e122 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:28.799508 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e52515-6e7c-40ca-9ce9-70a7b2f590ac service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:28.809253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e52515-6e7c-40ca-9ce9-70a7b2f590ac service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:28.863864 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:28.864685 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:28.864972 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:28.865250 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] PATH_INFO: `/v3/users/e328e4d952e74280abc9d1176a4aaf03` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:28.866450 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e328e4d952e74280abc9d1176a4aaf03)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:28.868379 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:29.056844 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e328e4d952e74280abc9d1176a4aaf03'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.058023 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e328e4d952e74280abc9d1176a4aaf03'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.058635 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e328e4d952e74280abc9d1176a4aaf03'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.065435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e328e4d952e74280abc9d1176a4aaf03'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.130030 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-9a865092-9275-4e93-ac8b-f411de1505b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e328e4d952e74280abc9d1176a4aaf03'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.147847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1424/2859] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:37:28 2022] DELETE /identity/v3/users/e328e4d952e74280abc9d1176a4aaf03 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:29.531701 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:29.533170 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:29.546408 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96adcd-358a-4a6f-bc53-b80d6fbb3767 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:29.601540 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:29.602444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:29.602733 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:29.603016 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] PATH_INFO: `/v3/projects/3caf8d8c8d1448379db98762afd4d155` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:29.604218 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3caf8d8c8d1448379db98762afd4d155)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:29.606666 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:29.748721 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3caf8d8c8d1448379db98762afd4d155', 'request_id': 'req-219d8862-91d1-4c93-8852-7ee6f44a6271'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.753987 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3caf8d8c8d1448379db98762afd4d155', 'request_id': 'req-219d8862-91d1-4c93-8852-7ee6f44a6271'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.823686 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3caf8d8c8d1448379db98762afd4d155', 'request_id': 'req-219d8862-91d1-4c93-8852-7ee6f44a6271'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.834944 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] The token cache is being invalidate because project 3caf8d8c8d1448379db98762afd4d155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:37:29.835235 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.835865 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-219d8862-91d1-4c93-8852-7ee6f44a6271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:29.837672 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1436/2860] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:37:29 2022] DELETE /identity/v3/projects/3caf8d8c8d1448379db98762afd4d155 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:29.843224 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb7649-640d-4286-a26f-acd41dd900ea None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:29.843730 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb7649-640d-4286-a26f-acd41dd900ea None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:29.843839 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb7649-640d-4286-a26f-acd41dd900ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:29.862529 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-71eb7649-640d-4286-a26f-acd41dd900ea None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:29.863776 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-71eb7649-640d-4286-a26f-acd41dd900ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:29.921858 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1425/2861] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:29.930891 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6420fa-3378-4a5d-81e0-59d84a5548d6 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:29.931174 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6420fa-3378-4a5d-81e0-59d84a5548d6 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:29.931458 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6420fa-3378-4a5d-81e0-59d84a5548d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:29.949507 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-ae6420fa-3378-4a5d-81e0-59d84a5548d6 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:29.950755 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-ae6420fa-3378-4a5d-81e0-59d84a5548d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:29.958626 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1437/2862] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:29.963010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb7649-640d-4286-a26f-acd41dd900ea None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:29.972928 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb7649-640d-4286-a26f-acd41dd900ea None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:29.977432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:29.978113 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:29.978392 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:29.978663 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:29.979417 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:37:29.980241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:29.981847 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:29.992772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1426/2863] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 13 13:37:29.996804 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6420fa-3378-4a5d-81e0-59d84a5548d6 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:30.005964 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6420fa-3378-4a5d-81e0-59d84a5548d6 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:30.011141 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b506577-7180-4665-8eb7-425aaa6dd5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:30.011811 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b506577-7180-4665-8eb7-425aaa6dd5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:30.012096 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b506577-7180-4665-8eb7-425aaa6dd5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:30.012376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b506577-7180-4665-8eb7-425aaa6dd5e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:30.013070 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b506577-7180-4665-8eb7-425aaa6dd5e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:30.015044 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b506577-7180-4665-8eb7-425aaa6dd5e4 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:30.111088 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1438/2864] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:37:29 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:37:30.114867 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:30.124544 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a56231-5c5c-4f62-a0df-21cd5a01521e admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:30.184474 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c9e0b-b67c-46d6-82cf-99c73e830d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:30.185144 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9e0b-b67c-46d6-82cf-99c73e830d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:30.185426 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9e0b-b67c-46d6-82cf-99c73e830d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:30.185705 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c9e0b-b67c-46d6-82cf-99c73e830d33 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:30.186432 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c9e0b-b67c-46d6-82cf-99c73e830d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:30.188347 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c9e0b-b67c-46d6-82cf-99c73e830d33 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:30.297969 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1427/2865] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:37:30 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:37:30.301967 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b506577-7180-4665-8eb7-425aaa6dd5e4 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:30.311952 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b506577-7180-4665-8eb7-425aaa6dd5e4 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:30.316469 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716429b8-ea10-4d33-b2bc-0ac819517d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:30.317119 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716429b8-ea10-4d33-b2bc-0ac819517d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:30.317405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716429b8-ea10-4d33-b2bc-0ac819517d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:30.317682 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716429b8-ea10-4d33-b2bc-0ac819517d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:30.318444 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716429b8-ea10-4d33-b2bc-0ac819517d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:30.320045 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716429b8-ea10-4d33-b2bc-0ac819517d96 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:30.330519 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1439/2866] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:37:30.334270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c9e0b-b67c-46d6-82cf-99c73e830d33 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:30.344960 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c9e0b-b67c-46d6-82cf-99c73e830d33 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:30.349505 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39960eb-d34a-49a6-8fb4-cd393d2c9cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:30.350253 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39960eb-d34a-49a6-8fb4-cd393d2c9cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:30.350541 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39960eb-d34a-49a6-8fb4-cd393d2c9cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:30.350820 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39960eb-d34a-49a6-8fb4-cd393d2c9cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:30.351546 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39960eb-d34a-49a6-8fb4-cd393d2c9cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:30.353125 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39960eb-d34a-49a6-8fb4-cd393d2c9cfe admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:30.362935 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1428/2867] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:37:30.366810 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716429b8-ea10-4d33-b2bc-0ac819517d96 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:30.376669 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716429b8-ea10-4d33-b2bc-0ac819517d96 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:30.382336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:30.382336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:30.382336 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:30.382688 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] PATH_INFO: `/v3/projects/aba143e9ba7d422d9288a7f12e84348e/users/b68eb6762c004f8dbc0de5d0cd3394d1/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:30.396734 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba143e9ba7d422d9288a7f12e84348e, user_id=b68eb6762c004f8dbc0de5d0cd3394d1, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:30.399440 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:30.492328 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1440/2868] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:37:30 2022] PUT /identity/v3/projects/aba143e9ba7d422d9288a7f12e84348e/users/b68eb6762c004f8dbc0de5d0cd3394d1/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 13 13:37:30.502934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2abb7e-fdc1-43f3-8391-60188efb62a6 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:30.503999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2abb7e-fdc1-43f3-8391-60188efb62a6 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:30.503999 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2abb7e-fdc1-43f3-8391-60188efb62a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:30.529794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-8f2abb7e-fdc1-43f3-8391-60188efb62a6 None None] MFA Rules not processed for user `b68eb6762c004f8dbc0de5d0cd3394d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:30.531011 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-8f2abb7e-fdc1-43f3-8391-60188efb62a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:30.592532 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1429/2869] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 13 13:37:30.603525 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:30.605914 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:30.660528 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50293895-9271-4405-9f4e-bdb4c738feaa admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:30.664099 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e6376-bab8-443a-ad5a-6def676c3e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:30.664728 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e6376-bab8-443a-ad5a-6def676c3e21 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:30.665009 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e6376-bab8-443a-ad5a-6def676c3e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:30.665283 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e6376-bab8-443a-ad5a-6def676c3e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:30.673900 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e6376-bab8-443a-ad5a-6def676c3e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:30.675322 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e6376-bab8-443a-ad5a-6def676c3e21 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:30.683736 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1441/2870] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:30.702767 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2abb7e-fdc1-43f3-8391-60188efb62a6 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:30.764494 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2abb7e-fdc1-43f3-8391-60188efb62a6 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:30.769071 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:30.769962 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:30.770242 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:30.770513 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] PATH_INFO: `/v3/users/b68eb6762c004f8dbc0de5d0cd3394d1` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:30.771693 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68eb6762c004f8dbc0de5d0cd3394d1)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:30.773601 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:30.955711 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68eb6762c004f8dbc0de5d0cd3394d1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:30.956346 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68eb6762c004f8dbc0de5d0cd3394d1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:30.956915 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68eb6762c004f8dbc0de5d0cd3394d1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:30.964633 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68eb6762c004f8dbc0de5d0cd3394d1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:31.103387 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68eb6762c004f8dbc0de5d0cd3394d1'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:31.120668 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1430/2871] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:37:30 2022] DELETE /identity/v3/users/b68eb6762c004f8dbc0de5d0cd3394d1 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:31.125368 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6169-f0c0-4feb-9b50-5d7169b71bce None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:31.125668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6169-f0c0-4feb-9b50-5d7169b71bce None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:31.125985 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e6169-f0c0-4feb-9b50-5d7169b71bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:31.142947 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-452e6169-f0c0-4feb-9b50-5d7169b71bce None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:31.146241 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-452e6169-f0c0-4feb-9b50-5d7169b71bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:31.210083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1442/2872] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:31.220599 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:31.222053 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:31.285167 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413f311-3d93-4f4a-bf3a-182dcff1eaa9 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:31.288772 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad1c0e-0ce1-4f61-a699-e7d8e291f119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:31.289424 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad1c0e-0ce1-4f61-a699-e7d8e291f119 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:31.289706 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad1c0e-0ce1-4f61-a699-e7d8e291f119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:31.290114 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad1c0e-0ce1-4f61-a699-e7d8e291f119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:31.300794 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad1c0e-0ce1-4f61-a699-e7d8e291f119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:31.305740 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad1c0e-0ce1-4f61-a699-e7d8e291f119 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:31.316025 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1431/2873] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:31.737870 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e6169-f0c0-4feb-9b50-5d7169b71bce None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:31.739390 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-452e6169-f0c0-4feb-9b50-5d7169b71bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:31.751954 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e6169-f0c0-4feb-9b50-5d7169b71bce None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:31.758920 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:31.759795 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:31.760072 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:31.760342 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] PATH_INFO: `/v3/projects/aba143e9ba7d422d9288a7f12e84348e` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:31.761763 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba143e9ba7d422d9288a7f12e84348e)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:31.763984 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:31.895867 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba143e9ba7d422d9288a7f12e84348e', 'request_id': 'req-8047fbd8-041c-439c-944e-3e683b9814e8'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:31.901018 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba143e9ba7d422d9288a7f12e84348e', 'request_id': 'req-8047fbd8-041c-439c-944e-3e683b9814e8'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:31.969933 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba143e9ba7d422d9288a7f12e84348e', 'request_id': 'req-8047fbd8-041c-439c-944e-3e683b9814e8'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:31.981703 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] The token cache is being invalidate because project aba143e9ba7d422d9288a7f12e84348e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:37:31.982002 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:31.982632 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-8047fbd8-041c-439c-944e-3e683b9814e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:31.984169 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1443/2874] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:37:31 2022] DELETE /identity/v3/projects/aba143e9ba7d422d9288a7f12e84348e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:31.989163 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551f73-ad09-4637-98c3-6b5d7f13ffec None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:31.989644 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551f73-ad09-4637-98c3-6b5d7f13ffec None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:31.989745 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551f73-ad09-4637-98c3-6b5d7f13ffec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.007625 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-84551f73-ad09-4637-98c3-6b5d7f13ffec None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:32.008822 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-84551f73-ad09-4637-98c3-6b5d7f13ffec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:32.064979 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1432/2875] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:32.072543 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bade5-543c-4fae-92c6-77622c0a5703 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.072832 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bade5-543c-4fae-92c6-77622c0a5703 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.073112 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bade5-543c-4fae-92c6-77622c0a5703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.091476 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-de6bade5-543c-4fae-92c6-77622c0a5703 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:32.092742 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-de6bade5-543c-4fae-92c6-77622c0a5703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:32.100936 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1444/2876] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:32.105565 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84551f73-ad09-4637-98c3-6b5d7f13ffec None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:32.115041 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84551f73-ad09-4637-98c3-6b5d7f13ffec None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:32.119641 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:32.120372 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.120659 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.120940 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] PATH_INFO: `/v3/domains` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.121671 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80277) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 13 13:37:32.122028 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:32.123629 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:32.135648 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1433/2877] 10.222.0.25 () {58 vars in 1152 bytes} [Fri May 13 13:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 13 13:37:32.140145 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bade5-543c-4fae-92c6-77622c0a5703 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:32.151681 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bade5-543c-4fae-92c6-77622c0a5703 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:32.156405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aeb94f-cf98-4ba6-a7b5-f165adff5d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:32.157061 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aeb94f-cf98-4ba6-a7b5-f165adff5d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.157348 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aeb94f-cf98-4ba6-a7b5-f165adff5d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.157628 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aeb94f-cf98-4ba6-a7b5-f165adff5d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.158929 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aeb94f-cf98-4ba6-a7b5-f165adff5d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:32.161420 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aeb94f-cf98-4ba6-a7b5-f165adff5d15 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:32.251124 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1445/2878] 10.222.0.25 () {60 vars in 1152 bytes} [Fri May 13 13:37:32 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 13 13:37:32.255265 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:32.265034 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f5ad-7544-4a46-9a31-6b511cb97578 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:32.319483 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0285ed-c0c9-4c5c-9393-bdd7c846a935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:32.320139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0285ed-c0c9-4c5c-9393-bdd7c846a935 admin admin] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.320446 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0285ed-c0c9-4c5c-9393-bdd7c846a935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.320719 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0285ed-c0c9-4c5c-9393-bdd7c846a935 admin admin] PATH_INFO: `/v3/users` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.321399 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0285ed-c0c9-4c5c-9393-bdd7c846a935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:32.323435 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0285ed-c0c9-4c5c-9393-bdd7c846a935 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:32.411934 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1434/2879] 10.222.0.25 () {60 vars in 1143 bytes} [Fri May 13 13:37:32 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 13 13:37:32.416011 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aeb94f-cf98-4ba6-a7b5-f165adff5d15 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:32.426189 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aeb94f-cf98-4ba6-a7b5-f165adff5d15 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:32.430759 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db884d27-d2a1-443f-b311-6b7aeff373e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:32.431416 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db884d27-d2a1-443f-b311-6b7aeff373e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.431741 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db884d27-d2a1-443f-b311-6b7aeff373e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.432020 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db884d27-d2a1-443f-b311-6b7aeff373e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.432973 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db884d27-d2a1-443f-b311-6b7aeff373e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:32.435405 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db884d27-d2a1-443f-b311-6b7aeff373e2 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:32.445668 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1446/2880] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:37:32.450139 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0285ed-c0c9-4c5c-9393-bdd7c846a935 admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:32.460010 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0285ed-c0c9-4c5c-9393-bdd7c846a935 admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:32.465978 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d90f22-9395-4fdb-b29a-796eafef4454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:32.466630 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d90f22-9395-4fdb-b29a-796eafef4454 admin admin] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.466918 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d90f22-9395-4fdb-b29a-796eafef4454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.467211 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d90f22-9395-4fdb-b29a-796eafef4454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.467961 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d90f22-9395-4fdb-b29a-796eafef4454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:32.469563 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d90f22-9395-4fdb-b29a-796eafef4454 admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:32.479743 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1435/2881] 10.222.0.25 () {58 vars in 1121 bytes} [Fri May 13 13:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 13 13:37:32.483819 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db884d27-d2a1-443f-b311-6b7aeff373e2 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:32.494925 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db884d27-d2a1-443f-b311-6b7aeff373e2 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:32.499894 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:32.500846 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.501252 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.501510 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] PATH_INFO: `/v3/projects/3c9c122ae3df4a6284114309af6ee2c7/users/422fc5c4d05548c295ec695632547432/roles/3552540dc0054d2e87fad160a358c58b` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.521881 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9c122ae3df4a6284114309af6ee2c7, user_id=422fc5c4d05548c295ec695632547432, role_id=3552540dc0054d2e87fad160a358c58b)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:32.524553 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:32.618259 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1447/2882] 10.222.0.25 () {60 vars in 1482 bytes} [Fri May 13 13:37:32 2022] PUT /identity/v3/projects/3c9c122ae3df4a6284114309af6ee2c7/users/422fc5c4d05548c295ec695632547432/roles/3552540dc0054d2e87fad160a358c58b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:32.629638 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134963e5-3870-4760-adc8-e2cc831ba9a0 None None] REQUEST_METHOD: `POST` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.631653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134963e5-3870-4760-adc8-e2cc831ba9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.631653 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134963e5-3870-4760-adc8-e2cc831ba9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.656823 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.auth.core [None req-134963e5-3870-4760-adc8-e2cc831ba9a0 None None] MFA Rules not processed for user `422fc5c4d05548c295ec695632547432`. Rule list: `[]` (Enabled: `True`). {{(pid=80277) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:32.658039 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-134963e5-3870-4760-adc8-e2cc831ba9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:32.717908 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1436/2883] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:32.728376 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:32.730079 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:32.783560 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33628808-1277-412e-a742-173e8ced9d27 admin admin] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:32.787308 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1184e19-1349-4b0a-a6ed-d21db685a493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7955ca85d92485d91abfa401b63f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:32.787958 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1184e19-1349-4b0a-a6ed-d21db685a493 service cinder] REQUEST_METHOD: `GET` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:32.788244 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1184e19-1349-4b0a-a6ed-d21db685a493 service cinder] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:32.788549 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1184e19-1349-4b0a-a6ed-d21db685a493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:32.796861 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1184e19-1349-4b0a-a6ed-d21db685a493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:32.798329 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1184e19-1349-4b0a-a6ed-d21db685a493 service cinder] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:32.806823 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1448/2884] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:43.944909 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134963e5-3870-4760-adc8-e2cc831ba9a0 None None] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:44.007917 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134963e5-3870-4760-adc8-e2cc831ba9a0 None None] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:44.012486 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:44.013359 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:44.013647 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:44.014609 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] PATH_INFO: `/v3/users/422fc5c4d05548c295ec695632547432` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:44.015804 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=422fc5c4d05548c295ec695632547432)` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:44.017769 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:44.213988 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '422fc5c4d05548c295ec695632547432'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:44.214564 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '422fc5c4d05548c295ec695632547432'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:44.215174 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '422fc5c4d05548c295ec695632547432'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:44.221831 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '422fc5c4d05548c295ec695632547432'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:44.291877 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.notifications [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '422fc5c4d05548c295ec695632547432'} {{(pid=80277) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:44.309655 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1437/2885] 10.222.0.25 () {58 vars in 1223 bytes} [Fri May 13 13:37:43 2022] DELETE /identity/v3/users/422fc5c4d05548c295ec695632547432 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:37:44.314951 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ae017-74ed-4212-a297-8214f0110bd9 None None] REQUEST_METHOD: `POST` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:44.315466 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ae017-74ed-4212-a297-8214f0110bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:44.315598 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ae017-74ed-4212-a297-8214f0110bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:44.331813 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.auth.core [None req-941ae017-74ed-4212-a297-8214f0110bd9 None None] MFA Rules not processed for user `df2ff43339cf407eb4cebe6484112e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80278) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 13 13:37:44.333001 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-941ae017-74ed-4212-a297-8214f0110bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:44.388186 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1449/2886] 10.222.0.25 () {58 vars in 957 bytes} [Fri May 13 13:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 13 13:37:44.397846 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:44.399334 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:44.455290 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365d8fa-b16c-45b2-82f9-b067ef287aad admin admin] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:44.459241 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ddb0dcaa5304016af1bc1f88845fb80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:44.459889 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:44.460175 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:44.460453 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:44.469700 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:44.471103 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:44.480568 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1438/2887] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 13 13:37:44.851664 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ae017-74ed-4212-a297-8214f0110bd9 None None] Authenticating user token {{(pid=80278) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:37:44.853083 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.fernet_utils [None req-941ae017-74ed-4212-a297-8214f0110bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80278) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:37:44.865582 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ae017-74ed-4212-a297-8214f0110bd9 None None] Validating token access rules against request {{(pid=80278) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:37:44.870178 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df2ff43339cf407eb4cebe6484112e01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '989de7565a074e299634e689ed15d9f9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80278) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:37:44.871042 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:37:44.871325 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:37:44.871609 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] PATH_INFO: `/v3/projects/3c9c122ae3df4a6284114309af6ee2c7` {{(pid=80278) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:37:44.872833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9c122ae3df4a6284114309af6ee2c7)` {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:37:44.874833 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] RBAC: Authorization granted {{(pid=80278) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:37:44.969580 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9c122ae3df4a6284114309af6ee2c7', 'request_id': 'req-65f8087b-6df5-44c3-b5a7-ea979fe33579'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:44.974785 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9c122ae3df4a6284114309af6ee2c7', 'request_id': 'req-65f8087b-6df5-44c3-b5a7-ea979fe33579'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:45.044171 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9c122ae3df4a6284114309af6ee2c7', 'request_id': 'req-65f8087b-6df5-44c3-b5a7-ea979fe33579'} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:45.058447 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] The token cache is being invalidate because project 3c9c122ae3df4a6284114309af6ee2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80278) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 13 13:37:45.058941 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:45.059684 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: DEBUG keystone.notifications [None req-65f8087b-6df5-44c3-b5a7-ea979fe33579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80278) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 13 13:37:45.061545 dv-cinder-841591-2-iscsi devstack@keystone.service[80278]: [pid: 80278|app: 0|req: 1450/2888] 10.222.0.25 () {58 vars in 1232 bytes} [Fri May 13 13:37:44 2022] DELETE /identity/v3/projects/3c9c122ae3df4a6284114309af6ee2c7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 13 13:38:19.062766 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] Authenticating user token {{(pid=80277) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 13 13:38:19.064217 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:38:19.154975 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a199a8-3359-4955-8070-8665c4aac847 service neutron] Validating token access rules against request {{(pid=80277) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 13 13:38:19.212351 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e7b35-00e7-4c7a-ba9f-7d4ff29a7462 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '310c5d3ba81044ac8f3f192728cc3e3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4bf72e0f9914e149849e2fa09294797', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80277) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 13 13:38:19.213270 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e7b35-00e7-4c7a-ba9f-7d4ff29a7462 service placement] REQUEST_METHOD: `GET` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 13 13:38:19.213555 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e7b35-00e7-4c7a-ba9f-7d4ff29a7462 service placement] SCRIPT_NAME: `/identity` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 13 13:38:19.213868 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e7b35-00e7-4c7a-ba9f-7d4ff29a7462 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80277) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 13 13:38:19.215678 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.fernet_utils [None req-576e7b35-00e7-4c7a-ba9f-7d4ff29a7462 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80277) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 13 13:38:19.226784 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e7b35-00e7-4c7a-ba9f-7d4ff29a7462 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 13 13:38:19.228380 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e7b35-00e7-4c7a-ba9f-7d4ff29a7462 service placement] RBAC: Authorization granted {{(pid=80277) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 13 13:38:19.238228 dv-cinder-841591-2-iscsi devstack@keystone.service[80277]: [pid: 80277|app: 0|req: 1439/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Fri May 13 13:38:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)